[{"_1":2,"_174":-5,"_175":-5},"loaderData",{"_3":4,"_13":14,"_21":22,"_153":154},"root",{"_5":-5,"_6":-7,"_7":8,"_9":-7,"_10":11,"_12":-7},"currentUser","primaryEmail","siteVerification","6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI","toast","enableDev",false,"incidentStatus","routes/_layout",{"_15":-5,"_16":17,"_18":17,"_19":11,"_20":11},"username","isAskGordonEnabled",true,"isUsingNewTemplate","showMinimalLanding","showHoneypotTrap","routes/_layout.r.$namespace.$reponame",{"_23":24,"_25":-5,"_26":11,"_27":11,"_28":29,"_30":31,"_32":-5,"_33":11,"_34":35,"_84":85,"_106":107,"_144":-5,"_145":11,"_146":17,"_147":11,"_148":-5,"_149":11,"_150":-5,"_151":152},"canonicalUrl","https://hub.docker.com/r/bde2020/mu-query-rewriter/tags","mcpServer","canSeeDockerfile","canSeeAutobuilds","namespace","bde2020","reponame","mu-query-rewriter","cloudBuildSource","isAutobuildsAvailable","repo",{"_36":37,"_38":39,"_41":42,"_43":44,"_45":11,"_46":47,"_48":49,"_50":51,"_53":31,"_28":29,"_54":55,"_56":40,"_57":58,"_59":60,"_61":62,"_63":-5,"_64":58,"_65":66,"_67":11,"_68":29,"_69":70,"_75":17,"_76":77,"_81":29,"_82":83},"categories",[],"content_types",[40],"image","date_registered","2017-12-21T12:10:59.856872Z","description","A proxy service for enriching and constraining SPARQL queries before they are sent to the db.","is_private","last_updated","2018-05-31T13:40:17.076587Z","last_modified","2024-10-16T13:48:34.145251Z","media_types",[52],"application/vnd.docker.container.image.v1+json","name","pull_count",1149,"repository_type","star_count",0,"status",1,"status_description","active","source","collaborator_count","full_description","# Mu Query Rewriter\n\nThe Query Rewriter is a proxy service for enriching and constraining SPARQL queries before they are sent to the database. It functions as an authorization service in the [mu-semtech](http://mu.semte.ch) microservice architecture, enabling in-database access control and authorization-aware caching.\n\nA sandbox interface for writing constraints is provided by https://github.com/big-data-europe/mu-query-rewriter-sandbox\n\nA basic working example and testing environment is provided by https://github.com/big-data-europe/graph-acl-basics/\n\n## Introduction \n\nA constraint is expressed as a standard SPARQL `CONSTRUCT` query, which conceptually represents an intermediate 'constraint' graph. An incoming query is optimally rewritten to a form which, when run against the full database, is equivalent to the original query being run against the constraint graph. Constraining queries in this way allows shared logic to be abstracted almost to the database level, simplifying the logic handled by each microservice. \n\n![rewriter diagram](rewriter.png)\n\nThe main use case is modeling access rights directly in the data, so that an incoming query is effectively run against the subset of data which the current user has permission to query or update. Using Annotations (see below), the Rewriter can return authorization-aware cache-keys and clear-keys to the mu-cache. When access rights can be fully resolved at rewrite-time (using functional properties and intermediate queries, see below), the rewriter can return an error signaling no access. When access can only be resolved in the database, an unauthorized query will return the empty set.\n\nThere are also simpler use cases, such as using multiple graphs to model data so that individual microservices do not need to be aware of the rules determining which triples are stored in which graph. \n\n### Examples\n\nIn the following example, the constraint defines a model where bikes and cars are stored in separate graphs, and users can be authorized to see one or both of the types. \n\nWhen a microservice in the mu-semtech architecture (so the identifier has assigned a `mu-session-id`) makes the query, the rewriter will send the rewritten query the database.\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n

Constraint Query Rewritten Query

Functional properties: rdf:type

Unique variables: ?user

Constraint	Query	Rewritten Query
Functional properties: `rdf:type`
Unique variables: `?user`
\n `\nCONSTRUCT {\n ?a ?b ?c\n}\nWHERE {\n GRAPH ?graph {\n ?a ?b ?c;\n a ?type\n }\n GRAPH <auth> {\n <SESSION> muauth:account ?user.\n ?user muauth:authorizedFor ?type\n }\n VALUES (?graph ?type){\n (<cars> <Car>)\n (<bikes> <Bike>)\n }\n}\n` \n	\n `\nSELECT *\nWHERE {\n ?s a <Bike>;\n <hasColor> ?color.\n}\n` \n	\n `\nSELECT ?s ?color\nWHERE {\n GRAPH ?graph23694 {\n ?s a <Bike>;\n <hasColor> ?color.\n }\n GRAPH <auth> {\n <session123456> muauth:account ?user.\n ?user muauth:authorizedFor <Bike>\n }\n VALUES (?graph23694) { (<bikes>) }\n}\n` \n

\nCONSTRUCT {\n  ?a ?b ?c\n}\nWHERE {\n  GRAPH ?graph {\n   ?a ?b ?c;\n      a ?type\n  }\n  GRAPH <auth> {\n   <SESSION> muauth:account ?user.\n   ?user muauth:authorizedFor ?type\n  }\n  VALUES (?graph ?type){\n    (<cars> <Car>)\n    (<bikes> <Bike>)\n  }\n}\n

\nSELECT *\nWHERE {\n  ?s a <Bike>;\n     <hasColor> ?color.\n}\n

\nSELECT ?s ?color\nWHERE {\n  GRAPH ?graph23694 {\n    ?s a <Bike>;\n       <hasColor> ?color.\n  }\n  GRAPH <auth> {\n   <session123456> muauth:account ?user.\n   ?user muauth:authorizedFor <Bike>\n  }\n  VALUES (?graph23694) { (<bikes>) }\n}\n

\n\nIf we want to query the database for `?user` at rewrite time, we declare `muauth:account` to be a transient functional property. (\"Transient\" means it is not cached between calls.) If `muauth:authorizedFor` is also functional and the user is not authorized to see ``s, this will be queried and known at rewrite time, and the query will fail before being sent to the database.\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n \n \n \n \n \n

Constraint	Query	Rewritten Query
Functional properties: `rdf:type`, `muauth:authorizedFor`
Transient functional properties: `muauth:account`
Unique variables: `?user`
\n `\nCONSTRUCT {\n ?a ?b ?c\n}\nWHERE {\n GRAPH ?graph {\n ?a ?b ?c;\n a ?type\n }\n GRAPH <auth> {\n <SESSION> mu:account ?user.\n ?user muauth:authorizedFor ?type\n }\n VALUES (?graph ?type){\n (<cars> <Car>)\n (<bikes> <Bike>)\n }\n}\n` \n	\n `\nSELECT *\nWHERE {\n ?s a <Bike>;\n <hasColor> ?color.\n}\n` \n	\n `\nSELECT ?s ?color\nWHERE {\n GRAPH ?graph23694 {\n ?s a <Bike>;\n <hasColor> ?color.\n }\n GRAPH <auth> {\n <session123456> mu:account <user4532>.\n <user4532> muauth:authorizedFor <Bike>\n }\n VALUES (?graph23694) { (<bikes>) }\n}\n` \n

\n\n## Running the Proxy Service\n\nThe Query Rewriter runs as a proxy service between the application and the database. It exposes a SPARQL endpoint `/sparql` that accepts GET and POST requests following the SPARQL specifications, and passes on all received headers to the database.\n\n### Configuration\n\nThe Query Rewriter supports the following environment variables:\n\n- `MU_SPARQL_ENDPOINT`: SPARQL read endpoint URL. Default: http://database:8890/sparql in Docker, and http://localhost:8890/sparql outside Docker.`.\n- `MU_SPARQL_UPDATE_ENDPOINT`: SPARQL update endpoint. Same defaults as preceding.\n- `PORT`: the port to run the application on, defaults to 8890.\n- `PLUGIN`: plugin filename (without '.scm' extension), to be loaded from `/config` in Docker and `./config/rewriter` locally.\n- `CACHE_QUERY_FORMS`: when \"true\" (default), will cache query forms. This feature is experimental (see below).\n- `QUERY_FUNCTIONAL_PROPERTIES`: when \"true\" (default), query the database for functional properties for known subjects.\n- `CALCULATE_ANNOTATIONS`: when \"true\" (default), annotations will be calculated and returned in the headers.\n- `QUERY_ANNOTATIONS`: when \"true\" (default), variable annotations will be queried in the database.\n- `SEND_DELTAS`: when \"true\" and a subscribers.json file is provided, will send deltas.\n- `DEBUG`: when \"true\", run Scheme code interpreted.\n- `DEBUG_LOGGING`: when \"true\", turn on verbose debug logging (mostly timing and performance).\n- `MESSAGE_LOGGING`: turns basic logging on or off.\n- `PRINT_SPARQL_QUERIES`: when \"true\", print all SPARQL queries.\n\nThese can also be set in the plugin file using the Scheme API below.\n\n\n### Example docker-compose file\n\n```\nversion: \"2\"\nservices:\n db:\n image: tenforce/virtuoso:1.0.0-virtuoso7.2.4\n environment:\n SPARQL_UPDATE: \"true\"\n DEFAULT_GRAPH: \"http://mu.semte.ch/application\"\n ports:\n - \"8890:8890\"\n volumes:\n - ./data/db:/data\n rewriter:\n image: nathanielrb/mu-graph-rewriter\n links:\n - db:database\n - laq:laq\n environment:\n DEBUG_LOGGING: \"true\"\n PLUGIN: \"authorization\"\n volumes:\n - ./config/rewriter:/config\n ports:\n - \"4027:8890\"\n my-service:\n image: my/service\n links:\n - rewriter:\n laq: # to test deltas\n image: flowofcontrol/list-all-requests\n\n```\n\n## Basic Logic\n\nA constraint is expressed as a SPARQL `CONSTRUCT` statement of one triple, called the \"matched triple\". The matched triple is matched against each triple in the incoming query, and the constraint's `WHERE` clause is rewritten with each match substitution, calculating minimal dependencies between the constrained variables to simplify the rewritten query.\n\nUnique variables are only rewritten once for the whole query, regardless of dependency relationships between variables.\n\nFunctional properties are unique, and if a subject has two values for a functional property in the same block, an error is signaled. When `QUERY_FUNCTIONAL_PROPERTIES` is \"true\", functional properties are queried in the database for known subjects.\n\nQueried properties are like functional properties but without the uniqueness restriction. For known subjects and objects, the triple is verified against the database:\n\n```\n ex:queriedProp \n```\n\n## Annotations\n\nAnnotations are used to define application-specific cache-keys and clear-keys for the mu-cache. They are defined as an extension to the SPARQL 1.1 standard, and can take two forms, constant annotations: `@access Label` and variable annotations: `@access Label(?var)`. \n\n```\n{\n ?a ?b ?c.\n ?a rdf:type ext:Comment.\n {\n @access adminComment\n ?user muauth:hasRole .\n}\nUNION\n{\n ?a ?b ?c.\n ?a rdf:type ?type.\n VALUES ?type { ext:Route ext:Hotel }\n {\n @access adminObject(?type)\n ?user muauth:hasRole .\n }\n```\n\nTwo headers are returned. `mu-cache-annotations` reports constant annotations, and variable annotations along with all possible values as known at rewrite time (not querying the database). `mu-queried-cache-annotations` reports actual values of variable annotations in the database.\n\n```\nMu-Cache-Annotations: \"adminComment,adminObject \"\nMu-Queried-Cache-Annotations: \"adminObject ,adminObject \"\n```\n\n## Deltas\n\nWhen the `SEND_DELTAS` parametre is \"true\" and a subscribers.json file is provided (see example in ./config/rewriter/subscribers.json), deltas are sent on all update queries.\n\nThe deltas are sent as JSON, following the format established by the [mu-delta-service](https://github.com/mu-semtech/mu-delta-service):\n\n```\n[\n {\n \"graph\":\"http://mu.semte.ch/application\",\n \"delta\": {\n \"inserts\":[\n {\n \"s\":\"http://data-hub.toerismevlaamsbrabant.be/hotels/5B0C1AA33C7DF9000C000003\",\n \"p\":\"http://mu.semte.ch/vocabularies/ext/addedBy\",\n \"o\":\"http://data-hub.toerismevlaamsbrabant.be/users/5B0C193C3C7DF9000C000001\"\n }\n ]\n }\n }\n]\n```\n\n## Cache Forms\n\nThe Rewriter comes with an experimental cache that caches the rewritten form of queries that are equivalent modulo full URIs and literal strings. In the current implementation, this is fairly risky, and cannot stand up to pathological (or even slightly wierd) cases. However, the speedup is considerable, and with proper precautions it is usable. A correct implementation is planned as a next step.\n\n\n\n\n\n## Limitations and Exceptions\n\nDue to the complexity of the SPARQL 1.1 grammar, not all SPARQL queries are fully supported.\n\nThe property paths `*`, `+` and `?` are constrained identically to the corresponding single-jump triple, e.g., `?s ?p* ?o` is considered subject to the same constraints as `?s ?p ?o`.\n\n## Writing Plugins\n\nThe [mu-query-writer-sandbox](https://github.com/big-data-europe/mu-query-rewriter-sandbox) provides a UI for writing and testing Query Rewriter plugins. The [graph-acl-basics](https://github.com/big-data-europe/graph-acl-basics/) repository provides a full working example for experimentation.\n\nThis section describes how to write plugins directly in Chicken Scheme. \n\n### Procedures\n\n**[procedure]** `(define-constraint mode constraint)` \n\n`mode` is a symbol, and can take the values `'read/write`, `'read` or `'write`\n\n`constraint` can be a string or a procedure of zero arguments (\"thunk\") returning a string.\n\n### Parameters\n\nMost of the parameters can be set as environment variables and in the sandbox, as described above (see ./framework/settings.scm). A few, however, can only be set in the Scheme code.\n\n**[parameter]** `*headers-replacements*`\n\nList of template forms for the constraint query that will be replaced dynamically with the matching header. Each element takes the form `'((\"

bde2020/mu-query-rewriter

Tags for bde2020/mu-query-rewriter