Benchmark for Pairs of Papers in Semantic Scholar: 1 hop vs. 2-4 hops version 0.0

Benchmark for Pairs of Papers in Semantic Scholar: 1 hop vs. 2-4 hops (version 0.0) There are two files: valid.txt and test.txt; both files use the same format. Columns 2 and 3 are corpus ids from Semantic Scholar. Column 1 is the distance between the two papers in the citation index. Columns 4 and 5 are the bins of the two paper, respectively. The bin is a number between 0 and 100. Papers are sorted by publication date. There are about 2M papers per bin, with the oldest papers in bin 0, and the newest papers in bin 99. Bin 100 is a catch-all for papers with unknown publication dates. head valid.txt 1 248518397 1041744 97 51 2 248518397 23848439 97 21 3 248518397 4235810 97 12 4 248518397 82079949 97 11 1 3374228 140728989 79 0 1 68334187 36144275 58 34 2 68334187 7008060 58 4 1 205881482 94036919 77 72 2 205881482 95069173 77 53 3 205881482 53480264 77 52 Each row is assigned to a bin, B, where B = max(col4, col5). Task: the task is to distinguish pairs of papers with distance == 1 from pairs of papers with distance > 1. Test/Train splits: For all thresholds, 0 <= T_{train} <= 99, train a model on rows in bins between 0 and T_{train} (inclusively). Test these models on rows in all bins 0 <= T_{test} <= 99. Report average accuracy for all combinations of T_{train} and T_{test}. Average Accuracy is defined as: mean(Predict(row) == 1, Gold(row) == 1) The means are computed over rows in a test bin.