AllegroGraph RDFStore Version 3.2 LUBM Benchmark Results
The following describes the performance of retrieval and RDFS reasoning with the native AllegroGraph RDFS++ Reasoner and AllegroGraph Prolog using the LUBM benchmark, with on average 20 departments per university.
These queries were performed with full RDFS++ reasoning at query time. Because the LUBM benchmark was designed to test some aspects of OWL reasoning that (by design) are beyond the strength of the RDFS++ reasoner, we added the single triple:
- ub:GraduateStudent rdfs:subClassOf ub:Student
The AllegroGraph RDFS++ Reasoner handles the following RDFS and OWL predicates correctly:
RDFS:
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://www.w3.org/2000/01/rdf-schema#subPropertyOf>
<http://www.w3.org/2000/01/rdf-schema#subClassOf>
<http://www.w3.org/2000/01/rdf-schema#range>
<http://www.w3.org/2000/01/rdf-schema#domain>
OWL:
<http://www.w3.org/2002/07/owl#sameAs>
<http://www.w3.org/2002/07/owl#inverseOf>
<http://www.w3.org/2002/07/owl#TransitiveProperty>
The AllegroGraph Reasoner does not handle full OWL restrictions. For that we recommend RacerPro.
Dynamic Materialization
AllegroGraph version 3.2's RDFS++ engine dynamically maintains the ontological entailments required for reasoning: it has no explicit materialization phase. Materialization is the pre-computation and storage of inferred triples so that future queries run more efficiently. The central problem with materialization is its maintenance: changes to the triple-store's ontology or facts usually change the set of inferred triples. In static materialization, any change in the store requires complete re-processing before new queries can run. AllegroGraph's dynamic materialization simplifies store maintenance and reduces the time required between data changes and querying.
AllegroGraph RDFStore 3.2 Benchmark for LUBM(50)
The total number of files read in is 1000. The total number of triples after running the queries is 6,875,705.
In the LUBM(50) results below, AllegroGraph's dynamic materialization occurred as necessary to answer each query. For AllegroGraph version 3.2 on System 1, loading, indexing and merging required a total of 7 minutes and 50 seconds. AllegroGraph version 3.2 on System 2, loading, indexing and merging required a total of 5 minutes and 21 seconds.
Query Performance Improvements in AllegroGraph version 3.2
Table 1 shows the results of running the LUBM 50 queries with both version 3.1 and 3.2 of AllegroGraph on System 1 running Linux with 16Gb of memory, and another run on version 3.2 on System 2 running Windows with 4Gb of memory. The total query time for the 14 queries on System 1 went from 275.379 seconds in version 3.1 to 3.684 seconds in version 3.2.
The results are reported in seconds. The total query time on System 2 was 2.022 seconds.
Table 1: Summary of LUBM(50) Results
Lubm Query |
# Triples |
3.1 Time - System 1 |
3.2 Time - System 1 |
3.2 Time - System 2 |
|---|---|---|---|---|
Query 1 |
4 |
0.000 |
0.006 |
0.015 |
Query 2 |
130 |
2.634 |
0.321 |
0.203 |
Query 3 |
6 |
0.002 |
0.006 |
0.001 |
Query 4 |
34 |
0.046 |
0.023 |
0.016 |
Query 5 |
719 |
3.899 |
0.055 |
0.032 |
Query 6 |
519,842 |
5.420 |
1.345 |
0.657 |
Query 7 |
67 |
0.027 |
0.012 |
0.001 |
Query 8 |
7,790 |
3.371 |
0.230 |
0.156 |
Query 9 |
13,639 |
254.107 |
1.203 |
0.704 |
Query 10 |
4 |
0.002 |
0.015 |
0.001 |
Query 11 |
224 |
0.075 |
0.021 |
0.016 |
Query 12 |
15 |
3.470 |
0.034 |
0.016 |
Query 13 |
228 |
0.091 |
0.013 |
0.001 |
Query 14 |
383,730 |
2.235 |
0.402 |
0.203 |
The first platform for the tests, System 1, was a Quad 1.8GHz AMD64 Opteron 884, with 16Gb of memory running Fedora 8.
The second platform for the test, System 2, was a 2.13GHz Intel Core Duo (6400), P5N-E SLl, nForce 650i SLl, with 4Gb of memory running 64-bit Windows XP Professional.
Download the LUBM(50) Benchmark files
The details are in the Learning Center example here: LUBM(50) Dataset
AllegroGraph RDFStore 3.2 Benchmark for LUBM(8000)
The total number of files read in is 160,007 N-Triples files, a total of 155 GB. The total number of triples after running the queries is 1,105,993,401. In the LUBM(8000) results below, AllegroGraph's dynamic materialization occurred as necessary to answer each query. For AllegroGraph version 3.2 on System 1, loading required a total of 30 hours, 28 minutes and 43.5 seconds. AllegroGraph version 3.2 on System 3, indexing and loading required a total of 22 hours, 28 minutes and 14.3 seconds. The total query time for System 1 was 9 minutes and 48.698 seconds. The total query time for System 3 was 5 minutes and 5.369 seconds.
Table 2 shows the results of running the LUBM(8000) queries with version 3.2 of AllegroGraph on 2 different machines. The results are reported in seconds.
Table 2: Summary of LUBM(8000) Results
Lubm Query |
# Triples |
Time - System 1 |
Time - System 3 |
|---|---|---|---|
Query 1 |
4 |
0.007 |
0.004 |
Query 2 |
2,528 |
112.500 |
54.964 |
Query 3 |
6 |
0.006 |
0.002 |
Query 4 |
34 |
0.024 |
0.014 |
Query 5 |
719 |
0.058 |
0.037 |
Query 6 |
83,557,706 |
197.976 |
104.794 |
Query 7 |
67 |
0.013 |
0.009 |
Query 8 |
7,790 |
0.491 |
0.178 |
Query 9 |
2,178,420 |
215.659 |
117.303 |
Query 10 |
4 |
0.009 |
0.005 |
Query 11 |
224 |
0.015 |
0.008 |
Query 12 |
15 |
0.024 |
0.014 |
Query 13 |
37,118 |
0.086 |
0.047 |
Query 14 |
63,400,587 |
61.825 |
27.990 |
The first platform for the tests, System 1, was a Quad 1.8GHz AMD64 Opteron 884, with 16Gb of memory running Fedora 8.
The second platform for the test, System 3, was a 2x Xeon E5410 (quad core) 2.33GHz, with 64Gb of memory.
Download the LUBM(8000) Benchmark files
The details are in the Learning Center example here: LUBM(8000) Dataset