From e27721b6eebd15897232d8d45b542aa0b1ba92eb Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 7 Nov 2014 01:28:51 -0500 Subject: [PATCH 001/121] v2 / API: core sections, incl. Cypher & transactions. --- API_v2.md | 222 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 API_v2.md diff --git a/API_v2.md b/API_v2.md new file mode 100644 index 0000000..ce9864c --- /dev/null +++ b/API_v2.md @@ -0,0 +1,222 @@ +# Node-Neo4j API v2 + +Scratchpad for designing a v2 of this library. + +For convenience, the code snippets below are written in +[CoffeeScript](http://coffeescript.org/) and +[Streamline](https://github.com/Sage/streamlinejs). +Essentially, underscores (`_`) represent standard Node.js `(result, error)` +callbacks, and "return" and "throw" refer to those callback args. + +These thoughts are written as scenarios followed by code snippets. + +Sections: + +- [General](#general) +- [Core](#core) +- [HTTP](#http) +- [Objects](#objects) +- [Cypher](#cypher) +- [Transactions](#transactions) + + +## General + +This driver will aim to generally be stateless and functional, +inspired by [React.js](http://facebook.github.io/react/). +Some context doesn't typically change, though (e.g. the URL to the database), +so this driver supports maintaining such context as simple state. + +This driver is geared towards the standard Node.js callback convention +mentioned above, but streams are now also returned wherever possible. +This allows both straightforward development using standard Node.js control +flow tools and libraries, while also supporting more advanced streaming usage. +You can freely use either, without worrying about the other. + +Importantly, if no callback is given (implying that the caller is streaming), +this driver will take care to not buffer any content in memory for callbacks, +ensuring high performance and low memory usage. + +This v2 driver converges on an options-/hash-based API for most/all methods. +This both conveys clearer intent and leaves room for future additions. + + +## Core + +**Let me make a "connection" to the database.** + +```coffee +neo4j = require 'neo4j' + +db = new neo4j.GraphDatabase + url: 'http://localhost:7474' + headers: ... # optional defaults, e.g. User-Agent + proxy: ... # optional +``` + +An upcoming version of Neo4j will likely add native authentication. +We already support HTTP Basic Auth in the URL, but we may then need to add +ways to manage the auth (e.g. generate and reset tokens). + +The current v1 of the driver is hypermedia-driven, so it discovers the +`/db/data` endpoint. We may hardcode that in v2 for efficiency and simplicity, +but if we do, do we need to make that customizable/overridable too? + + +## HTTP + +**Let me make arbitrary HTTP requests to the REST API.** + +This will allow callers to make any API requests; no one will be blocked by +this driver not supporting a particular API. + +It'll also allow callers to interface with arbitrary plugins, +including custom ones. + +```coffee +db.http {method, path, headers, body}, _ +``` + +This method will immediately return the raw, pipeable HTTP response stream, +and "return" (via callback) the full HTTP response when it's finished. +The body will be parsed as JSON, and nodes and relationships will be +transformed into `Node` and `Relationship` objects (see below). + +Importantly, we don't want to leak the implementation details of which HTTP +library we use. Both [request](https://github.com/request/request) and +[SuperAgent](http://visionmedia.github.io/superagent/#piping-data) are great; +it'd be nice to experiment with both (e.g. SuperAgent supports the browser). +Does this mean we should do anything special when returning HTTP responses? +E.g. should we document our own minimal HTTP `Response` interface that's the +common subset of both libraries? + +Also worth asking: what about streaming the response JSON? +It looks like [Oboe.js](http://oboejs.com/) supports reading an existing HTTP +stream ([docs](http://oboejs.com/api#byo-stream)), but not in the browser. +Is that fine? + + +## Objects + +**Give me useful objects, instead of raw JSON.** + +Transform the Neo4j REST API's raw JSON format for nodes and relationships +into useful `Node` and `Relationship` objects. +These objects don't have to do anything / have any mutating methods; +they're just container objects that serve to organize information. + +```coffee +class Node {_id, labels, properties} +class Relationship {_id, type, properties, _fromId, _toId} +``` + +TODO: Transform path JSON into `Path` objects too? +We have this in v1, but is it really useful and functional these days? +E.g. see [issue #57](https://github.com/thingdom/node-neo4j/issues/57). + +Importantly, using Neo4j's native IDs is strongly discouraged these days, +so v2 of this driver makes that an explicit private property. +It's still there if you need it, e.g. for the old-school traversal API. + +That extends to relationships: they explicitly do *not* link to `Node` +*instances* anymore (since that data is frequently not available), only IDs. +Those IDs are thus private too. + +Also importantly, there's no more notion of persistence or updating +(e.g. no more `save()` method) in this v2 API. +If you want to update data and persist it back to Neo4j, +you do that the same way as you would without these objects. +*This driver is not an ORM/OGM. +Those problems must be solved separately.* + +It's similarly no longer possible to create or get an instance of either of +these classes that represents data that isn't persisted yet (like the current +v1's *synchronous* `db.createNode()` method returns). +These classes are only instantiated internally, and only returned async'ly +from database responses. + + +## Cypher + +**Let me make simple, parametrized Cypher queries.** + +```coffee +db.cypher {query, params, headers, raw}, _ +``` + +This method will immediately return a pipeable "results" stream (a `data` +event will be emitted for each result row), then "return" (via callback) +the full results array at the end (similar to the `http()` method). +Each result row will be a dictionary from column name to the row's value for +that column. + +By default, nodes and relationships will be transformed to `Node` and +`Relationship` objects. +To do that, though, requires a heavier data format over the wire. +If you don't need the full knowledge of node and relationship metadata +(labels, types, native IDs), you can bypass this by passing `raw: true` +for a potential performance gain. + +If there's an error, the "results" stream will emit an `error` event, +as well as "throw" (via callback) the error. + +TODO: Should we formalize the "results" stream into a documented class? + +TODO: Should we allow access to other underlying data formats, e.g. "graph"? + + +## Transactions + +**Let me make multiple queries, across multiple network requests, +all within a single transaction.** + +This is the trickiest part of the API to design. +I've tried my best to design this using the use cases we have at FiftyThree, +but it's very hard to know whether this is designed well for a broader set of +use cases without having more experience or feedback. + +Example use case: complex delete. +I want to delete an image, which has some image-specific business logic, +but in addition, I need to delete any likes and comments on the image. +Each of those has its own specific business logic (which may also be +recursive), so our code can't capture everything in a single query. +Thus, we need to make one query to delete the comments and likes (which may +actually be multiple queries, as well), then a second one to delete the image. +We want to do all of that transactionally, so that if any one query fails, +we abort/rollback and either retry or report failure to the user. + +Given a use case like that, this API is optimized for **one query per network +request**, *not* multiple queries per network request ("batching"). +I *think* batching is always an optimization (never a true *requirement*), +so it could always be achieved automatically under-the-hood by this driver +(e.g. by waiting until the next event loop tick to send the actual queries). +Please provide feedback if you disagree! + +```coffee +tx = db.beginTransaction {...} # any options needed? +``` + +This method returns a `Transaction` object, which mainly just encapsulates the +state of a "transaction ID" returned by Neo4j from the first query. + +This method is named "begin" instead of "create" to reflect that it returns +immediately, and has not actually persisted anything to the database yet. + +```coffee +class Transaction {_id} + +tx.cypher {query, params, headers, raw, commit}, _ +tx.commit _ +tx.rollback _ +``` + +The transactional `cypher` method is just like the regular `cypher` method, +except that it supports an additional `commit` option, which can be set to +`true` to automatically attempt to commit the transaction after this query. + +Otherwise, transactions can be committed and rolled back independently. + +TODO: Any more functionality needed for transactions? +There's a notion of expiry, and the expiry timeout can be reset by making +empty queries; should a notion of auto "renewal" (effectively, a higher +timeout than the default) be built-in for convenience? From 3c9ab24afaa06d2b09c44c6b7ee66c1d4809ddfc Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 7 Nov 2014 02:30:23 -0500 Subject: [PATCH 002/121] v2 / API: semantic errors! --- API_v2.md | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/API_v2.md b/API_v2.md index ce9864c..0cab74b 100644 --- a/API_v2.md +++ b/API_v2.md @@ -18,6 +18,7 @@ Sections: - [Objects](#objects) - [Cypher](#cypher) - [Transactions](#transactions) +- [Errors](#errors) ## General @@ -220,3 +221,96 @@ TODO: Any more functionality needed for transactions? There's a notion of expiry, and the expiry timeout can be reset by making empty queries; should a notion of auto "renewal" (effectively, a higher timeout than the default) be built-in for convenience? + + +## Errors + +**Throw meaningful and semantic errors.** + +Background reading — a huge source of inspiration that informs much of the +API design here: + +http://www.joyent.com/developers/node/design/errors + +Neo4j v2 provides excellent error info for (transactional) Cypher requests: + +http://neo4j.com/docs/stable/status-codes.html + +Essentially, errors are grouped into three "classifications": +**client errors**, **database errors**, and **transient errors**. +There are additional "categories" and "titles", but the high-level +classifications are just the right level of granularity for decision-making +(e.g. whether to convey the error to the user, fail fast, or retry). + +```json +{ + "code": "Neo.ClientError.Statement.EntityNotFound", + "message": "Node with id 741073" +} +``` + +Unfortunately, other endpoints return errors in a completely different format +and style. E.g.: + +- [404 `NodeNotFoundException`](http://neo4j.com/docs/stable/rest-api-nodes.html#rest-api-get-non-existent-node) +- [409 `OperationFailureException`](http://neo4j.com/docs/stable/rest-api-nodes.html#rest-api-nodes-with-relationships-cannot-be-deleted) +- [400 `PropertyValueException`](http://neo4j.com/docs/stable/rest-api-node-properties.html#rest-api-property-values-can-not-be-null) +- [400 `BadInputException` w/ nested `ConstraintViolationException` and `IllegalTokenNameException`](http://neo4j.com/docs/stable/rest-api-node-labels.html#rest-api-adding-a-label-with-an-invalid-name) + +```json +{ + "exception": "BadInputException", + "fullname": "org.neo4j.server.rest.repr.BadInputException", + "message": "Unable to add label, see nested exception.", + "stacktrace": [ + "org.neo4j.server.rest.web.DatabaseActions.addLabelToNode(DatabaseActions.java:328)", + "org.neo4j.server.rest.web.RestfulGraphDatabase.addNodeLabel(RestfulGraphDatabase.java:447)", + "java.lang.reflect.Method.invoke(Method.java:606)", + "org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)", + "java.lang.Thread.run(Thread.java:744)" + ], + "cause": {...} +} +``` + +One important distinction is that (transactional) Cypher errors *don't* have +any associated HTTP status code (since the results are streamed), +while the "legacy" exceptions do. +Fortunately, HTTP 4xx and 5xx status codes map almost directly to +"client error" and "database error" classifications, while +"transient" errors can be detected by name. + +So when it comes to designing this driver's v2 error API, +there are two open questions: + +1. Should this driver abstract away this discrepancy in Neo4j error formats, + and present a uniform error API across the two? + Or should it expose these two different formats? + +2. Should this driver return standard `Error` objects decorated w/ e.g. a + `neo4j` property? Or should it define its own `Error` subclasses? + +The current design of this API chooses to present a uniform (but minimal) API +using `Error` subclasses. Importantly: + +- The subclasses correspond to the client/database/transient classifications + mentioned above, for easy decision-making via either the `instanceof` + operator or the `name` property. + +- Special care is taken to provide `message` and `stack` properties rich in + info, so that no special serialization is needed to debug production errors. + +- And all info returned by Neo4j is also available on the `Error` instances + under a `neo4j` property, for deeper introspection and analysis if desired. + +```coffee +class Error {name, message, stack, neo4j} + +class ClientError extends Error +class DatabaseError extends Error +class TransientError extends Error +``` + +TODO: Should we name these classes with a `Neo4j` prefix? +They'll only be exposed via this driver's `module.exports`, so it's not +technically necessary, but that'd allow for e.g. destructuring. From 702abc0777343f6dec44fae4d0cdcb1bbcdfafbf Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 7 Nov 2014 03:07:20 -0500 Subject: [PATCH 003/121] v2 / API: schema! Labels, indexes, constraints. --- API_v2.md | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/API_v2.md b/API_v2.md index 0cab74b..0ebea75 100644 --- a/API_v2.md +++ b/API_v2.md @@ -19,6 +19,7 @@ Sections: - [Cypher](#cypher) - [Transactions](#transactions) - [Errors](#errors) +- [Schema](#schema) ## General @@ -314,3 +315,79 @@ class TransientError extends Error TODO: Should we name these classes with a `Neo4j` prefix? They'll only be exposed via this driver's `module.exports`, so it's not technically necessary, but that'd allow for e.g. destructuring. + + +## Schema + +**Let me manage the database schema: labels, indexes, and constraints.** + +Much of this is already possible with Cypher, but a few things aren't +(e.g. listing all labels). +Even for the things that are, this driver can provide convenience methods +for those boilerplate Cypher queries. + +### Labels + +```coffee +db.getLabels _ +``` + +Methods to manipulate labels on nodes are purposely *not* provided, +as those start to get into the territory of operations that should really be +performed atomically within a Cypher query. +(E.g. labels should generally be set on nodes right when they're created.) +Conveniences for those kinds of things are perfect for an ORM/OGM to solve. + +Labels are simple strings. + +### Indexes + +```coffee +db.getIndexes _ # across all labels +db.getIndexes {label}, _ # for a particular label +db.hasIndex {label, property}, _ +db.createIndex {label, property}, _ +db.deleteIndex {label, property}, _ +``` + +Returned indexes are minimal `Index` objects: + +```coffee +class Index {label, property} +``` + +TODO: Neo4j's REST API actually takes and returns *arrays* of properties, +but AFAIK, all indexes today only deal with a single property. +Should multiple properties be supported? + +### Constraints + +The only constraint type implemented by Neo4j today is the uniqueness +constraint, so this API defaults to that. +The design aims to be generic in order to support future constraint types, +but it's still possible that the API may have to break when that happens. + +```coffee +db.getConstraints _ # across all labels +db.getConstraints {label}, _ # for a particular label +db.hasConstraint {label, property}, _ +db.createConstraint {label, property}, _ +db.deleteConstraint {label, property}, _ +``` + +Returned constraints are minimal `Constraint` objects: + +```coffee +class Constraint {label, type, property} +``` + +TODO: Neo4j's REST API actually takes and returns *arrays* of properties, +but uniqueness constraints today only deal with a single property. +Should multiple properties be supported? + +### Misc + +```coffee +db.getPropertyKeys _ +db.getRelationshipTypes _ +``` From 15ca10e41328c6b1b1d8cea01193c7b42299208b Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 7 Nov 2014 04:08:59 -0500 Subject: [PATCH 004/121] v2 / API: legacy indexing! Simple usage, uniqueness, auto-indexes. --- API_v2.md | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/API_v2.md b/API_v2.md index 0ebea75..fd765e9 100644 --- a/API_v2.md +++ b/API_v2.md @@ -20,6 +20,7 @@ Sections: - [Transactions](#transactions) - [Errors](#errors) - [Schema](#schema) +- [Legacy Indexing](#legacy-indexing) ## General @@ -391,3 +392,110 @@ Should multiple properties be supported? db.getPropertyKeys _ db.getRelationshipTypes _ ``` + + +## Legacy Indexing + +Neo4j v2's constraint-based indexing has yet to implement much of the +functionality provided by Neo4j v1's legacy indexing (e.g. relationships, +support for arrays, fulltext indexing). +This driver thus provides legacy indexing APIs. + +### Management + +```coffee +db.getLegacyNodeIndexes _ +db.getLegacyNodeIndex {name}, _ +db.createLegacyNodeIndex {name, config}, _ +db.deleteLegacyNodeIndex {name}, _ + +db.getLegacyRelationshipIndexes _ +db.getLegacyRelationshipIndex {name}, _ +db.createLegacyRelationshipIndex {name, config}, _ +db.deleteLegacyRelationshipIndex {name}, _ +``` + +Both returned legacy node indexes and legacy relationship indexes are +minimal `LegacyIndex` objects: + +```coffee +class LegacyIndex {name, config} +``` + +The `config` property is e.g. `{provider: 'lucene', type: 'fulltext'}`; +[full documentation here](http://neo4j.com/docs/stable/indexing-create-advanced.html). + +### Simple Usage + +```coffee +db.addNodeToLegacyIndex {name, key, value, _id} +db.getNodesFromLegacyIndex {name, key, value} # key-value lookup +db.getNodesFromLegacyIndex {name, query} # arbitrary Lucene query +db.removeNodeFromLegacyIndex {name, key, value, _id} # key, value optional + +db.addRelationshipToLegacyIndex {name, key, value, _id} +db.getRelationshipsFromLegacyIndex {name, key, value} # key-value lookup +db.getRelationshipsFromLegacyIndex {name, query} # arbitrary Lucene query +db.removeRelationshipFromLegacyIndex {name, key, value, _id} # key, value optional +``` + +### Uniqueness + +Neo4j v1's legacy indexing provides a uniqueness constraint for +adding (existing) and creating (new) nodes and relationships. +It has two modes: + +- "Get or create": if an existing node or relationship is found for the given + key and value, return it, otherwise add this node or relationship + (creating it if it's new). + +- "Create or fail": if an existing node or relationship is found for the given + key and value, fail, otherwise add this node or relationship + (creating it if it's new). + +For adding existing nodes or relationships, simply pass `unique: true` to the +`add` method. + +```coffee +db.addNodeToLegacyIndex {name, key, value, _id, unique: true} +db.addRelationshipToLegacyIndex {name, key, value, _id, unique: true} +``` + +This uses the "create or fail" mode. +It's hard to imagine a real-world use case for "get or create" when adding +existing nodes, but please offer feedback if you have one. + +For creating new nodes or relationships, the `create` method below corresponds +with "create or fail", while `getOrCreate` corresponds with "get or create": + +```coffee +db.createNodeFromLegacyIndex {name, key, value, properties} +db.getOrCreateNodeFromLegacyIndex {name, key, value, properties} + +db.createRelationshipFromLegacyIndex {name, key, value, type, properties, _fromId, _toId} +db.getOrCreateRelationshipFromLegacyIndex {name, key, value, type, properties, _fromId, _toId} +``` + +### Auto-Indexes + +Neo4j provides two automatic legacy indexes: +`node_auto_index` and `relationship_auto_index`. +Instead of hardcoding those index names in your app, +Neo4j provides separate legacy auto-indexing APIs, +which this driver exposes as well. + +The APIs are effectively the same as the above; +just replace `LegacyIndex` with `LegacyAutoIndex` in all method names, +then omit the `name` parameter. + +```coffee +db.getNodesFromLegacyAutoIndex {key, value} # key-value lookup +db.getNodesFromLegacyAutoIndex {query} # arbitrary Lucene query +db.createNodeFromLegacyAutoIndex {key, value, properties} +db.getOrCreateNodeFromLegacyAutoIndex {key, value, properties} + +db.getRelationshipsFromLegacyAutoIndex {key, value} # key-value lookup +db.getRelationshipsFromLegacyAutoIndex {query} # arbitrary Lucene query +db.createRelationshipFromLegacyAutoIndex {key, value, type, properties, _fromId, _toId} +db.getOrCreateRelationshipFromLegacyAutoIndex {key, value, type, properties, _fromId, _toId} +``` From 38e8f9023a99a23369bc04816ef1da961612b89a Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 9 Nov 2014 15:03:29 -0500 Subject: [PATCH 005/121] v2 / API: update spec to plain JavaScript. --- API_v2.md | 222 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 128 insertions(+), 94 deletions(-) diff --git a/API_v2.md b/API_v2.md index fd765e9..4570e14 100644 --- a/API_v2.md +++ b/API_v2.md @@ -1,16 +1,6 @@ # Node-Neo4j API v2 -Scratchpad for designing a v2 of this library. - -For convenience, the code snippets below are written in -[CoffeeScript](http://coffeescript.org/) and -[Streamline](https://github.com/Sage/streamlinejs). -Essentially, underscores (`_`) represent standard Node.js `(result, error)` -callbacks, and "return" and "throw" refer to those callback args. - -These thoughts are written as scenarios followed by code snippets. - -Sections: +This is a rough, work-in-progress redesign of the node-neo4j API. - [General](#general) - [Core](#core) @@ -30,31 +20,34 @@ inspired by [React.js](http://facebook.github.io/react/). Some context doesn't typically change, though (e.g. the URL to the database), so this driver supports maintaining such context as simple state. -This driver is geared towards the standard Node.js callback convention -mentioned above, but streams are now also returned wherever possible. +This driver continues to accept standard Node.js callbacks of the form +`function (result, error)`, but streams are now also returned where possible. This allows both straightforward development using standard Node.js control -flow tools and libraries, while also supporting more advanced streaming usage. +flow tools and libraries, while also supporting more advanced streaming uses. You can freely use either, without worrying about the other. -Importantly, if no callback is given (implying that the caller is streaming), -this driver will take care to not buffer any content in memory for callbacks, +Importantly, callbacks can be omitted when streaming, and in that case, +this driver will take special care not to buffer results in memory, ensuring high performance and low memory usage. This v2 driver converges on an options-/hash-based API for most/all methods. This both conveys clearer intent and leaves room for future additions. +For the sake of this rough spec, you can assume most options are optional, +but this will be documented more precisely in the final API documentation. ## Core **Let me make a "connection" to the database.** -```coffee -neo4j = require 'neo4j' +```js +var neo4j = require('neo4j'); -db = new neo4j.GraphDatabase - url: 'http://localhost:7474' - headers: ... # optional defaults, e.g. User-Agent - proxy: ... # optional +var db = new neo4j.GraphDatabase({ + url: 'http://localhost:7474', + headers: {}, // optional defaults, e.g. User-Agent + proxy: '', // optional +}); ``` An upcoming version of Neo4j will likely add native authentication. @@ -76,12 +69,14 @@ this driver not supporting a particular API. It'll also allow callers to interface with arbitrary plugins, including custom ones. -```coffee -db.http {method, path, headers, body}, _ +```js +function cb(err, resp) {}; + +var req = db.http({method, path, headers, body}, cb); ``` -This method will immediately return the raw, pipeable HTTP response stream, -and "return" (via callback) the full HTTP response when it's finished. +This method will immediately return the raw, pipeable HTTP request stream, +then call the callback with the full HTTP response when it's finished. The body will be parsed as JSON, and nodes and relationships will be transformed into `Node` and `Relationship` objects (see below). @@ -143,15 +138,18 @@ from database responses. **Let me make simple, parametrized Cypher queries.** -```coffee -db.cypher {query, params, headers, raw}, _ +```js +function cb(err, results) {}; + +var stream = db.cypher({query, params, headers, raw}, cb); ``` -This method will immediately return a pipeable "results" stream (a `data` -event will be emitted for each result row), then "return" (via callback) -the full results array at the end (similar to the `http()` method). -Each result row will be a dictionary from column name to the row's value for -that column. +This method will immediately return a pipeable "results" stream +(a `data` event will be emitted for each result row), +then call the callback with the the full, aggregate results array +(similar to the `http()` method). +Each result row will be a dictionary from column name +to the row's value for that column. By default, nodes and relationships will be transformed to `Node` and `Relationship` objects. @@ -161,7 +159,7 @@ If you don't need the full knowledge of node and relationship metadata for a potential performance gain. If there's an error, the "results" stream will emit an `error` event, -as well as "throw" (via callback) the error. +and the callback will be called with the error. TODO: Should we formalize the "results" stream into a documented class? @@ -195,8 +193,8 @@ so it could always be achieved automatically under-the-hood by this driver (e.g. by waiting until the next event loop tick to send the actual queries). Please provide feedback if you disagree! -```coffee -tx = db.beginTransaction {...} # any options needed? +```js +var tx = db.beginTransaction(); // any options needed? ``` This method returns a `Transaction` object, which mainly just encapsulates the @@ -207,10 +205,16 @@ immediately, and has not actually persisted anything to the database yet. ```coffee class Transaction {_id} +``` + +```js +function cbResults(err, results) {}; +function cbDone(err) {}; + +var stream = tx.cypher({query, params, headers, raw, commit}, cbResults); -tx.cypher {query, params, headers, raw, commit}, _ -tx.commit _ -tx.rollback _ +tx.commit(cbDone); +tx.rollback(cbDone); ``` The transactional `cypher` method is just like the regular `cypher` method, @@ -329,8 +333,10 @@ for those boilerplate Cypher queries. ### Labels -```coffee -db.getLabels _ +```js +function cb(err, labels) {}; + +db.getLabels(cb); ``` Methods to manipulate labels on nodes are purposely *not* provided, @@ -343,12 +349,17 @@ Labels are simple strings. ### Indexes -```coffee -db.getIndexes _ # across all labels -db.getIndexes {label}, _ # for a particular label -db.hasIndex {label, property}, _ -db.createIndex {label, property}, _ -db.deleteIndex {label, property}, _ +```js +function cbOne(err, index) {}; +function cbMany(err, indexes) {}; +function cbBool(err, bool) {}; +function cbDone(err) {}; + +db.getIndexes(cbMany); // across all labels +db.getIndexes({label}, cbMany); // for a particular label +db.hasIndex({label, property}, cbBool); +db.createIndex({label, property}, cbOne); +db.deleteIndex({label, property}, cbDone); ``` Returned indexes are minimal `Index` objects: @@ -368,12 +379,17 @@ constraint, so this API defaults to that. The design aims to be generic in order to support future constraint types, but it's still possible that the API may have to break when that happens. -```coffee -db.getConstraints _ # across all labels -db.getConstraints {label}, _ # for a particular label -db.hasConstraint {label, property}, _ -db.createConstraint {label, property}, _ -db.deleteConstraint {label, property}, _ +```js +function cbOne(err, constraint) {}; +function cbMany(err, constraints) {}; +function cbBool(err, bool) {}; +function cbDone(err) {}; + +db.getConstraints(cbMany); // across all labels +db.getConstraints({label}, cbMany); // for a particular label +db.hasConstraint({label, property}, cbBool); +db.createConstraint({label, property}, cbOne); +db.deleteConstraint({label, property}, cbDone); ``` Returned constraints are minimal `Constraint` objects: @@ -388,9 +404,12 @@ Should multiple properties be supported? ### Misc -```coffee -db.getPropertyKeys _ -db.getRelationshipTypes _ +```js +function cbKeys(err, keys) {}; +function cbTypes(err, types) {}; + +db.getPropertyKeys(cbKeys); +db.getRelationshipTypes(cbTypes); ``` @@ -403,16 +422,20 @@ This driver thus provides legacy indexing APIs. ### Management -```coffee -db.getLegacyNodeIndexes _ -db.getLegacyNodeIndex {name}, _ -db.createLegacyNodeIndex {name, config}, _ -db.deleteLegacyNodeIndex {name}, _ - -db.getLegacyRelationshipIndexes _ -db.getLegacyRelationshipIndex {name}, _ -db.createLegacyRelationshipIndex {name, config}, _ -db.deleteLegacyRelationshipIndex {name}, _ +```js +function cbOne(err, index) {}; +function cbMany(err, indexes) {}; +function cbDone(err) {}; + +db.getLegacyNodeIndexes(cbMany); +db.getLegacyNodeIndex({name}, cbOne); +db.createLegacyNodeIndex({name, config}, cbOne); +db.deleteLegacyNodeIndex({name}, cbDone); + +db.getLegacyRelationshipIndexes(cbMany); +db.getLegacyRelationshipIndex({name}, cbOne); +db.createLegacyRelationshipIndex({name, config}, cbOne); +db.deleteLegacyRelationshipIndex({name}, cbDone); ``` Both returned legacy node indexes and legacy relationship indexes are @@ -427,16 +450,20 @@ The `config` property is e.g. `{provider: 'lucene', type: 'fulltext'}`; ### Simple Usage -```coffee -db.addNodeToLegacyIndex {name, key, value, _id} -db.getNodesFromLegacyIndex {name, key, value} # key-value lookup -db.getNodesFromLegacyIndex {name, query} # arbitrary Lucene query -db.removeNodeFromLegacyIndex {name, key, value, _id} # key, value optional - -db.addRelationshipToLegacyIndex {name, key, value, _id} -db.getRelationshipsFromLegacyIndex {name, key, value} # key-value lookup -db.getRelationshipsFromLegacyIndex {name, query} # arbitrary Lucene query -db.removeRelationshipFromLegacyIndex {name, key, value, _id} # key, value optional +```js +function cbOne(err, node_or_rel) {}; +function cbMany(err, nodes_or_rels) {}; +function cbDone(err) {}; + +db.addNodeToLegacyIndex({name, key, value, _id}, cbOne); +db.getNodesFromLegacyIndex({name, key, value}, cbMany); // key-value lookup +db.getNodesFromLegacyIndex({name, query}, cbMany); // arbitrary Lucene query +db.removeNodeFromLegacyIndex({name, key, value, _id}, cbDone); // key, value optional + +db.addRelationshipToLegacyIndex({name, key, value, _id}, cbOne); +db.getRelationshipsFromLegacyIndex({name, key, value}, cbMany); // key-value lookup +db.getRelationshipsFromLegacyIndex({name, query}, cbMany); // arbitrary Lucene query +db.removeRelationshipFromLegacyIndex({name, key, value, _id}, cbDone); // key, value optional ``` ### Uniqueness @@ -456,9 +483,11 @@ It has two modes: For adding existing nodes or relationships, simply pass `unique: true` to the `add` method. -```coffee -db.addNodeToLegacyIndex {name, key, value, _id, unique: true} -db.addRelationshipToLegacyIndex {name, key, value, _id, unique: true} +```js +function cb(err, node_or_rel) {}; + +db.addNodeToLegacyIndex({name, key, value, _id, unique: true}, cb); +db.addRelationshipToLegacyIndex({name, key, value, _id, unique: true}, cb); ``` This uses the "create or fail" mode. @@ -468,12 +497,14 @@ existing nodes, but please offer feedback if you have one. For creating new nodes or relationships, the `create` method below corresponds with "create or fail", while `getOrCreate` corresponds with "get or create": -```coffee -db.createNodeFromLegacyIndex {name, key, value, properties} -db.getOrCreateNodeFromLegacyIndex {name, key, value, properties} +```js +function cb(err, node_or_rel) {}; + +db.createNodeFromLegacyIndex({name, key, value, properties}, cb); +db.getOrCreateNodeFromLegacyIndex({name, key, value, properties}, cb); -db.createRelationshipFromLegacyIndex {name, key, value, type, properties, _fromId, _toId} -db.getOrCreateRelationshipFromLegacyIndex {name, key, value, type, properties, _fromId, _toId} +db.createRelationshipFromLegacyIndex({name, key, value, type, properties, _fromId, _toId}, cb); +db.getOrCreateRelationshipFromLegacyIndex({name, key, value, type, properties, _fromId, _toId}, cb); ``` ### Auto-Indexes @@ -488,14 +519,17 @@ The APIs are effectively the same as the above; just replace `LegacyIndex` with `LegacyAutoIndex` in all method names, then omit the `name` parameter. -```coffee -db.getNodesFromLegacyAutoIndex {key, value} # key-value lookup -db.getNodesFromLegacyAutoIndex {query} # arbitrary Lucene query -db.createNodeFromLegacyAutoIndex {key, value, properties} -db.getOrCreateNodeFromLegacyAutoIndex {key, value, properties} - -db.getRelationshipsFromLegacyAutoIndex {key, value} # key-value lookup -db.getRelationshipsFromLegacyAutoIndex {query} # arbitrary Lucene query -db.createRelationshipFromLegacyAutoIndex {key, value, type, properties, _fromId, _toId} -db.getOrCreateRelationshipFromLegacyAutoIndex {key, value, type, properties, _fromId, _toId} +```js +function cbOne(err, node_or_rel) {}; +function cbMany(err, nodes_or_rels) {}; + +db.getNodesFromLegacyAutoIndex({key, value}, cbMany); // key-value lookup +db.getNodesFromLegacyAutoIndex({query}, cbMany); // arbitrary Lucene query +db.createNodeFromLegacyAutoIndex({key, value, properties}, cbOne); +db.getOrCreateNodeFromLegacyAutoIndex({key, value, properties}, cbOne); + +db.getRelationshipsFromLegacyAutoIndex({key, value}, cbMany); // key-value lookup +db.getRelationshipsFromLegacyAutoIndex({query}, cbMany); // arbitrary Lucene query +db.createRelationshipFromLegacyAutoIndex({key, value, type, properties, _fromId, _toId}, cbOne); +db.getOrCreateRelationshipFromLegacyAutoIndex({key, value, type, properties, _fromId, _toId}, cbOne); ``` From e9c1cb1cef58d8cbf4359e538d9c4791b3ad8c4b Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 9 Nov 2014 18:07:04 -0500 Subject: [PATCH 006/121] v2 / API: change streaming, HTTP, Cypher. --- API_v2.md | 69 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 36 insertions(+), 33 deletions(-) diff --git a/API_v2.md b/API_v2.md index 4570e14..fc72d61 100644 --- a/API_v2.md +++ b/API_v2.md @@ -21,14 +21,13 @@ Some context doesn't typically change, though (e.g. the URL to the database), so this driver supports maintaining such context as simple state. This driver continues to accept standard Node.js callbacks of the form -`function (result, error)`, but streams are now also returned where possible. -This allows both straightforward development using standard Node.js control -flow tools and libraries, while also supporting more advanced streaming uses. -You can freely use either, without worrying about the other. +`function (result, error)`, for straightforward development using standard +Node.js control flow tools and libraries. -Importantly, callbacks can be omitted when streaming, and in that case, -this driver will take special care not to buffer results in memory, -ensuring high performance and low memory usage. +In addition, however, this driver also supports streaming use cases now. +Where specified, callbacks can be omitted to have get back streams instead. +And importantly, in that case, this driver will take special care not to +buffer results in memory, ensuring high performance and low memory usage. This v2 driver converges on an options-/hash-based API for most/all methods. This both conveys clearer intent and leaves room for future additions. @@ -70,15 +69,26 @@ It'll also allow callers to interface with arbitrary plugins, including custom ones. ```js -function cb(err, resp) {}; +function cb(err, body) {}; -var req = db.http({method, path, headers, body}, cb); +var req = db.http({method, path, headers, body, raw}, cb); ``` -This method will immediately return the raw, pipeable HTTP request stream, -then call the callback with the full HTTP response when it's finished. -The body will be parsed as JSON, and nodes and relationships will be -transformed into `Node` and `Relationship` objects (see below). +This method will immediately return a duplex HTTP stream, to and from which +both request and response body data can be piped or streamed. + +In addition, if a callback is given, it will be called with the final result. +By default, this result will be the HTTP response body (parsed as JSON), +with nodes and relationships transformed to +[`Node` and `Relationship` objects](#objects), +or an [`Error` object](#errors) both if there was a native error (e.g. DNS) +or if the HTTP status code is `4xx` or `5xx`. + +Alternately, `raw: true` can be passed to have the callback receive the full +HTTP response, with `statusCode`, `headers`, and `body` properties. +The `body` will still be parsed as JSON, but nodes, relationships, and errors +will *not* be transformed to node-neo4j objects in this case. +In addition, `4xx` and `5xx` status code will *not* yield an error. Importantly, we don't want to leak the implementation details of which HTTP library we use. Both [request](https://github.com/request/request) and @@ -88,11 +98,6 @@ Does this mean we should do anything special when returning HTTP responses? E.g. should we document our own minimal HTTP `Response` interface that's the common subset of both libraries? -Also worth asking: what about streaming the response JSON? -It looks like [Oboe.js](http://oboejs.com/) supports reading an existing HTTP -stream ([docs](http://oboejs.com/api#byo-stream)), but not in the browser. -Is that fine? - ## Objects @@ -144,24 +149,22 @@ function cb(err, results) {}; var stream = db.cypher({query, params, headers, raw}, cb); ``` -This method will immediately return a pipeable "results" stream -(a `data` event will be emitted for each result row), -then call the callback with the the full, aggregate results array -(similar to the `http()` method). -Each result row will be a dictionary from column name -to the row's value for that column. +If a callback is given, it'll be called with the array of results, +or an error if there is one. +Alternately, the results can be streamed back by omitting the callback. +In that case, a stream will be returned, which will emit a `data` event +for each result, or an `error` event if there is one. +In both cases, each result will be a dictionary from column name to +row value for that column. -By default, nodes and relationships will be transformed to `Node` and -`Relationship` objects. -To do that, though, requires a heavier data format over the wire. +In addition, by default, nodes and relationships will be transformed to +`Node` and `Relationship` objects. If you don't need the full knowledge of node and relationship metadata -(labels, types, native IDs), you can bypass this by passing `raw: true` -for a potential performance gain. - -If there's an error, the "results" stream will emit an `error` event, -and the callback will be called with the error. +(labels, types, native IDs), you can bypass that by specifying `raw: true`, +which will return just property data, for a potential performance gain. -TODO: Should we formalize the "results" stream into a documented class? +TODO: Should we formalize the streaming case into a documented Stream class? +Or should we just link to cypher-stream? TODO: Should we allow access to other underlying data formats, e.g. "graph"? From 58e953aa76294c3bc925f00f831aae56da71b6d9 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 9 Nov 2014 21:29:27 -0500 Subject: [PATCH 007/121] v2 / Misc: update package.json dev deps. --- package.json | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/package.json b/package.json index 82d77a7..8b1e66f 100644 --- a/package.json +++ b/package.json @@ -14,10 +14,10 @@ "request": "^2.27.0" }, "devDependencies": { - "chai": "^1.8.0", - "codo": "^1.5.0", - "coffee-script": "1.7.x", - "mocha": "^1.3.0", + "chai": "^1.9.2", + "codo": "^2.0.9", + "coffee-script": "1.8.x", + "mocha": "^2.0.1", "streamline": "^0.10.16" }, "engines": { From e75340d82e02f5939d55566b5afac6d810c995ad Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 22 Jan 2015 12:57:04 -0500 Subject: [PATCH 008/121] v2 / Misc: package.json updates, e.g. version. --- package.json | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/package.json b/package.json index 8b1e66f..9f4100c 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,10 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "1.1.1", - "author": "The Thingdom ", + "version": "2.0.0-alpha1", + "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", - "Aseem Kishore ", "Sergio Haro " ], "main": "./lib", @@ -21,7 +20,7 @@ "streamline": "^0.10.16" }, "engines": { - "node": ">= 0.6" + "node": ">= 0.10" }, "scripts": { "build": "coffee -m -c lib/ && _coffee -m --standalone -c lib/", @@ -32,7 +31,7 @@ "test": "mocha" }, "keywords": [ - "neo4j", "graph", "database", "driver", "rest", "api", "client" + "neo4j", "graph", "database", "driver", "client", "cypher" ], "license": "Apache-2.0", "repository" : { From e208ecdfd5442c08b1f4c8aa991197f692a53b63 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 22 Jan 2015 13:01:38 -0500 Subject: [PATCH 009/121] v2 / Tests: move existing tests to different dir. Temporarily; for now. That'll present a cleaner diff. --- {test => test-old}/crud._coffee | 0 {test => test-old}/cypher._coffee | 0 {test => test-old}/gremlin._coffee | 0 {test => test-old}/misc._coffee | 0 4 files changed, 0 insertions(+), 0 deletions(-) rename {test => test-old}/crud._coffee (100%) rename {test => test-old}/cypher._coffee (100%) rename {test => test-old}/gremlin._coffee (100%) rename {test => test-old}/misc._coffee (100%) diff --git a/test/crud._coffee b/test-old/crud._coffee similarity index 100% rename from test/crud._coffee rename to test-old/crud._coffee diff --git a/test/cypher._coffee b/test-old/cypher._coffee similarity index 100% rename from test/cypher._coffee rename to test-old/cypher._coffee diff --git a/test/gremlin._coffee b/test-old/gremlin._coffee similarity index 100% rename from test/gremlin._coffee rename to test-old/gremlin._coffee diff --git a/test/misc._coffee b/test-old/misc._coffee similarity index 100% rename from test/misc._coffee rename to test-old/misc._coffee From 5474d3af658ebd01ee9d6f48b4d1cb353527b7be Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 22 Jan 2015 13:02:11 -0500 Subject: [PATCH 010/121] v2 / Misc: move existing code to different dir. For now, to present a cleaner diff. --- .codoopts | 2 +- {lib => lib-old}/GraphDatabase._coffee | 0 {lib => lib-old}/Node._coffee | 0 {lib => lib-old}/Path.coffee | 0 {lib => lib-old}/PropertyContainer._coffee | 0 {lib => lib-old}/Relationship._coffee | 0 {lib => lib-old}/index.coffee | 0 {lib => lib-old}/util.coffee | 0 package.json | 6 +++--- 9 files changed, 4 insertions(+), 4 deletions(-) rename {lib => lib-old}/GraphDatabase._coffee (100%) rename {lib => lib-old}/Node._coffee (100%) rename {lib => lib-old}/Path.coffee (100%) rename {lib => lib-old}/PropertyContainer._coffee (100%) rename {lib => lib-old}/Relationship._coffee (100%) rename {lib => lib-old}/index.coffee (100%) rename {lib => lib-old}/util.coffee (100%) diff --git a/.codoopts b/.codoopts index e5e1dca..3e0d9af 100644 --- a/.codoopts +++ b/.codoopts @@ -1,4 +1,4 @@ --name 'Node-Neo4j API Documentation' --title 'Node-Neo4j API Documentation' --readme API.md -./lib +./lib-old diff --git a/lib/GraphDatabase._coffee b/lib-old/GraphDatabase._coffee similarity index 100% rename from lib/GraphDatabase._coffee rename to lib-old/GraphDatabase._coffee diff --git a/lib/Node._coffee b/lib-old/Node._coffee similarity index 100% rename from lib/Node._coffee rename to lib-old/Node._coffee diff --git a/lib/Path.coffee b/lib-old/Path.coffee similarity index 100% rename from lib/Path.coffee rename to lib-old/Path.coffee diff --git a/lib/PropertyContainer._coffee b/lib-old/PropertyContainer._coffee similarity index 100% rename from lib/PropertyContainer._coffee rename to lib-old/PropertyContainer._coffee diff --git a/lib/Relationship._coffee b/lib-old/Relationship._coffee similarity index 100% rename from lib/Relationship._coffee rename to lib-old/Relationship._coffee diff --git a/lib/index.coffee b/lib-old/index.coffee similarity index 100% rename from lib/index.coffee rename to lib-old/index.coffee diff --git a/lib/util.coffee b/lib-old/util.coffee similarity index 100% rename from lib/util.coffee rename to lib-old/util.coffee diff --git a/package.json b/package.json index 9f4100c..dacaab7 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,7 @@ "Daniel Gasienica ", "Sergio Haro " ], - "main": "./lib", + "main": "./lib-old", "dependencies": { "http-status": "^0.1.0", "request": "^2.27.0" @@ -23,8 +23,8 @@ "node": ">= 0.10" }, "scripts": { - "build": "coffee -m -c lib/ && _coffee -m --standalone -c lib/", - "clean": "rm -f lib/*.{js,map}", + "build": "coffee -m -c lib-old/ && _coffee -m --standalone -c lib-old/", + "clean": "rm -f lib-old/*.{js,map}", "codo": "codo && codo --server", "prepublish": "npm run build", "postpublish": "npm run clean", From ef90612814bfb17dc6814cbbfe97d9c17b5bb438 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 23 Jan 2015 21:52:55 -0500 Subject: [PATCH 011/121] v2 / Tests: flesh out; begin core tests! --- package.json | 2 +- test-new/README.md | 33 +++++++++++ test-new/core._coffee | 134 ++++++++++++++++++++++++++++++++++++++++++ test/mocha.opts | 1 - 4 files changed, 168 insertions(+), 2 deletions(-) create mode 100644 test-new/README.md create mode 100644 test-new/core._coffee diff --git a/package.json b/package.json index dacaab7..1956430 100644 --- a/package.json +++ b/package.json @@ -28,7 +28,7 @@ "codo": "codo && codo --server", "prepublish": "npm run build", "postpublish": "npm run clean", - "test": "mocha" + "test": "mocha test-new" }, "keywords": [ "neo4j", "graph", "database", "driver", "client", "cypher" diff --git a/test-new/README.md b/test-new/README.md new file mode 100644 index 0000000..28f5246 --- /dev/null +++ b/test-new/README.md @@ -0,0 +1,33 @@ +## Node-Neo4j Tests + +Many of these tests are written in [Streamline.js](https://github.com/Sage/streamlinejs) syntax, for convenience and robustness. + +In a nutshell, instead of calling async functions with a callback (that takes an error and a result), we get to call them with an `_` parameter instead, and then pretend as if the functions are synchronous (*returning* their results and *throwing* any errors). + +E.g. instead of writing tests like this: + +```coffee +it 'should get foo then set bar', (done) -> + db.getFoo (err, foo) -> + expect(err).to.not.exist() + expect(foo).to.be.a 'number' + + db.setBar foo, (err, bar) -> + expect(err).to.not.exist() + expect(bar).to.equal foo + + done() +``` + +We get to write tests like this: + +```coffee +it 'should get foo then set bar', (_) -> + foo = db.getFoo _ + expect(foo).to.be.a 'number' + + bar = db.setBar foo, _ + expect(bar).to.equal foo +``` + +This lets us write more concise tests that simultaneously test for errors more thoroughly. (If an async function "throws" an error, the test will fail.) diff --git a/test-new/core._coffee b/test-new/core._coffee new file mode 100644 index 0000000..63559ac --- /dev/null +++ b/test-new/core._coffee @@ -0,0 +1,134 @@ +{expect} = require 'chai' +{GraphDatabase} = require '../' +http = require 'http' + + +# SHARED STATE + +DB = null +URL = process.env.NEO4J_URL or 'http://localhost:7474' + +FAKE_PROXY = 'http://lorem.ipsum' +FAKE_HEADERS = + 'x-foo': 'bar-baz' + 'x-lorem': 'ipsum' + # TODO: Test custom User-Agent behavior? + + +## TESTS + +describe 'GraphDatabase::constructor', -> + + it 'should support full options', -> + DB = new GraphDatabase + url: URL + proxy: FAKE_PROXY + headers: FAKE_HEADERS + + expect(DB).to.be.an.instanceOf GraphDatabase + expect(DB.url).to.equal URL + expect(DB.proxy).to.equal FAKE_PROXY + expect(DB.headers).to.be.an 'object' + + # Default headers should include/contain our given ones, + # but may include extra default headers too (e.g. X-Stream): + for key, val of FAKE_HEADERS + expect(DB.headers[key]).to.equal val + + it 'should support just URL string', -> + DB = new GraphDatabase URL + + expect(DB).to.be.an.instanceOf GraphDatabase + expect(DB.url).to.equal URL + expect(DB.proxy).to.not.exist() + expect(DB.headers).to.be.an 'object' + + it 'should throw if no URL given', -> + fn = -> new GraphDatabase() + expect(fn).to.throw TypeError + + # Also try giving an options argument, just with no URL: + fn = -> new GraphDatabase {proxy: FAKE_PROXY} + expect(fn).to.throw TypeError + +describe 'GraphDatabase::http', -> + + it 'should support simple GET requests by default', (_) -> + body = DB.http '/', _ + + expect(body).to.be.an 'object' + expect(body).to.have.keys 'data', 'management' + + it 'should support complex requests with options', (_) -> + body = DB.http + method: 'GET' + path: '/' + headers: FAKE_HEADERS + , _ + + expect(body).to.be.an 'object' + expect(body).to.have.keys 'data', 'management' + + it 'should throw errors for 4xx responses by default', (_) -> + try + thrown = false + DB.http + method: 'POST' + path: '/' + , _ + catch err + thrown = true + expect(err).to.be.an.instanceOf Error + # TODO: Deeper and more semantic assertions, e.g. status code. + + expect(thrown).to.be.true() + + it 'should support returning raw responses', (_) -> + resp = DB.http + method: 'GET' + path: '/' + raw: true + , _ + + expect(resp).to.be.an.instanceOf http.IncomingMessage + + {statusCode, headers, body} = resp + + expect(statusCode).to.equal 200 + + expect(headers).to.be.an 'object' + expect(headers['content-type']).to.match /// ^application/json\b /// + + expect(body).to.be.an 'object' + expect(body).to.have.keys 'data', 'management' + + it 'should not throw 4xx errors for raw responses', (_) -> + resp = DB.http + method: 'POST' + path: '/' + raw: true + , _ + + expect(resp).to.be.an.instanceOf http.IncomingMessage + expect(resp.statusCode).to.equal 405 # Method Not Allowed + + it 'should throw native errors always', (_) -> + db = new GraphDatabase 'http://idontexist.foobarbaz.nodeneo4j' + + try + thrown = false + db.http + path: '/' + raw: true + , _ + catch err + thrown = true + expect(err).to.be.an.instanceOf Error + # TODO: Deeper and more semantic assertions? + + expect(thrown).to.be.true() + + it 'should support streaming' + # Test that it immediately returns a duplex HTTP stream. + # Test writing request data to this stream. + # Test reading response data from this stream. diff --git a/test/mocha.opts b/test/mocha.opts index 653a2f1..80e0c18 100644 --- a/test/mocha.opts +++ b/test/mocha.opts @@ -1,5 +1,4 @@ --compilers coffee:coffee-script/register,_coffee:streamline/register ---ui exports --reporter spec --timeout 10000 --slow 1000 From da2f6e073e511d074b566585aff7afc902731067 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 23 Jan 2015 21:55:05 -0500 Subject: [PATCH 012/121] v2 / Core: begin new lib, GraphDatabase class. --- lib-new/GraphDatabase.coffee | 68 ++++++++++++++++++++++++++++++++++++ lib-new/index.coffee | 1 + package.json | 10 +++--- 3 files changed, 74 insertions(+), 5 deletions(-) create mode 100644 lib-new/GraphDatabase.coffee create mode 100644 lib-new/index.coffee diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee new file mode 100644 index 0000000..10f76fa --- /dev/null +++ b/lib-new/GraphDatabase.coffee @@ -0,0 +1,68 @@ +$ = require 'underscore' +lib = require '../package.json' +Request = require 'request' +URL = require 'url' + +module.exports = class GraphDatabase + + # Default HTTP headers: + headers: + 'User-Agent': "node-neo4j/#{lib.version}" + 'X-Stream': 'true' + + constructor: (opts={}) -> + if typeof opts is 'string' + opts = {url: opts} + + {@url, @headers, @proxy} = opts + + if not @url + throw new TypeError 'URL to Neo4j required' + + # TODO: Do we want to special-case User-Agent? Blacklist X-Stream? + @headers or= {} + $(@headers).defaults @constructor::headers + + http: (opts={}, cb) -> + if typeof opts is 'string' + opts = {path: opts} + + {method, path, headers, body, raw} = opts + + if not path + throw new TypeError 'Path required' + + method or= 'GET' + headers or= {} + + # TODO: Parse response body before calling back. + # TODO: Do we need to do anything special to support streaming response? + req = Request + method: method + url: URL.resolve @url, path + headers: $(headers).defaults @headers + json: body ? true + , (err, resp) => + + if err + # TODO: Properly check error and transform to semantic instance. + return cb err + + if raw + # TODO: Do we want to return our own Response object? + return cb null, resp + + {body, headers, statusCode} = resp + + if statusCode >= 500 + # TODO: Semantic errors. + err = new Error "Neo4j #{statusCode} response" + return cb err + + if statusCode >= 400 + # TODO: Semantic errors. + err = new Error "Neo4j #{statusCode} response" + return cb err + + # TODO: Parse nodes and relationships. + return cb null, body diff --git a/lib-new/index.coffee b/lib-new/index.coffee new file mode 100644 index 0000000..b7f319c --- /dev/null +++ b/lib-new/index.coffee @@ -0,0 +1 @@ +exports.GraphDatabase = require './GraphDatabase' diff --git a/package.json b/package.json index 1956430..120fbd0 100644 --- a/package.json +++ b/package.json @@ -7,10 +7,10 @@ "Daniel Gasienica ", "Sergio Haro " ], - "main": "./lib-old", + "main": "./lib-new", "dependencies": { - "http-status": "^0.1.0", - "request": "^2.27.0" + "request": "^2.27.0", + "underscore": "1.7.x" }, "devDependencies": { "chai": "^1.9.2", @@ -23,8 +23,8 @@ "node": ">= 0.10" }, "scripts": { - "build": "coffee -m -c lib-old/ && _coffee -m --standalone -c lib-old/", - "clean": "rm -f lib-old/*.{js,map}", + "build": "coffee -m -c lib-new/", + "clean": "rm -f lib-new/*.{js,map}", "codo": "codo && codo --server", "prepublish": "npm run build", "postpublish": "npm run clean", From 2f8ad1a3f9ceda61153b9d313c48e02ec6205aa8 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 23 Jan 2015 23:21:20 -0500 Subject: [PATCH 013/121] v2 / Tests: improve/DRY up core tests. --- test-new/core._coffee | 99 ++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 38 deletions(-) diff --git a/test-new/core._coffee b/test-new/core._coffee index 63559ac..29ede30 100644 --- a/test-new/core._coffee +++ b/test-new/core._coffee @@ -1,6 +1,7 @@ +$ = require 'underscore' {expect} = require 'chai' -{GraphDatabase} = require '../' http = require 'http' +neo4j = require '../' # SHARED STATE @@ -12,7 +13,50 @@ FAKE_PROXY = 'http://lorem.ipsum' FAKE_HEADERS = 'x-foo': 'bar-baz' 'x-lorem': 'ipsum' - # TODO: Test custom User-Agent behavior? + # TODO: Test overlap with default headers? + # TODO: Test custom User-Agent behavior, or blacklist X-Stream? + + +## HELPERS + +# +# Asserts that the given object is an instance of GraphDatabase, +# pointing to the given URL, optionally using the given proxy URL. +# +expectDatabase = (db, url, proxy) -> + expect(db).to.be.an.instanceOf neo4j.GraphDatabase + expect(db.url).to.equal url + expect(db.proxy).to.equal proxy + +# +# Asserts that the given GraphDatabase instance has its `headers` property set +# to the union of the given headers and the default GraphDatabase `headers`, +# with the given headers taking precedence. +# +# TODO: If we special-case User-Agent or blacklist X-Stream, update here. +# +expectHeaders = (db, headers) -> + expect(db.headers).to.be.an 'object' + + defaultHeaders = neo4j.GraphDatabase::headers + defaultKeys = Object.keys defaultHeaders + givenKeys = Object.keys headers + expectedKeys = $.union defaultKeys, givenKeys # This de-dupes too. + + # This is an exact check, i.e. *only* these keys: + expect(db.headers).to.have.keys expectedKeys + + for key, val of db.headers + expect(val).to.equal headers[key] or defaultHeaders[key] + +expectResponse = (resp, statusCode) -> + expect(resp).to.be.an.instanceOf http.IncomingMessage + expect(resp.statusCode).to.equal statusCode + expect(resp.headers).to.be.an 'object' + +expectNeo4jRoot = (body) -> + expect(body).to.be.an 'object' + expect(body).to.have.keys 'data', 'management' ## TESTS @@ -20,44 +64,33 @@ FAKE_HEADERS = describe 'GraphDatabase::constructor', -> it 'should support full options', -> - DB = new GraphDatabase + DB = new neo4j.GraphDatabase url: URL proxy: FAKE_PROXY headers: FAKE_HEADERS - expect(DB).to.be.an.instanceOf GraphDatabase - expect(DB.url).to.equal URL - expect(DB.proxy).to.equal FAKE_PROXY - expect(DB.headers).to.be.an 'object' - - # Default headers should include/contain our given ones, - # but may include extra default headers too (e.g. X-Stream): - for key, val of FAKE_HEADERS - expect(DB.headers[key]).to.equal val + expectDatabase DB, URL, FAKE_PROXY + expectHeaders DB, FAKE_HEADERS it 'should support just URL string', -> - DB = new GraphDatabase URL + DB = new neo4j.GraphDatabase URL - expect(DB).to.be.an.instanceOf GraphDatabase - expect(DB.url).to.equal URL - expect(DB.proxy).to.not.exist() - expect(DB.headers).to.be.an 'object' + expectDatabase DB, URL + expectHeaders DB, {} it 'should throw if no URL given', -> - fn = -> new GraphDatabase() + fn = -> new neo4j.GraphDatabase() expect(fn).to.throw TypeError # Also try giving an options argument, just with no URL: - fn = -> new GraphDatabase {proxy: FAKE_PROXY} + fn = -> new neo4j.GraphDatabase {proxy: FAKE_PROXY} expect(fn).to.throw TypeError describe 'GraphDatabase::http', -> it 'should support simple GET requests by default', (_) -> body = DB.http '/', _ - - expect(body).to.be.an 'object' - expect(body).to.have.keys 'data', 'management' + expectNeo4jRoot body it 'should support complex requests with options', (_) -> body = DB.http @@ -66,8 +99,7 @@ describe 'GraphDatabase::http', -> headers: FAKE_HEADERS , _ - expect(body).to.be.an 'object' - expect(body).to.have.keys 'data', 'management' + expectNeo4jRoot body it 'should throw errors for 4xx responses by default', (_) -> try @@ -90,17 +122,9 @@ describe 'GraphDatabase::http', -> raw: true , _ - expect(resp).to.be.an.instanceOf http.IncomingMessage - - {statusCode, headers, body} = resp - - expect(statusCode).to.equal 200 - - expect(headers).to.be.an 'object' - expect(headers['content-type']).to.match /// ^application/json\b /// - - expect(body).to.be.an 'object' - expect(body).to.have.keys 'data', 'management' + expectResponse resp, 200 + expect(resp.headers['content-type']).to.match /// ^application/json\b /// + expectNeo4jRoot resp.body it 'should not throw 4xx errors for raw responses', (_) -> resp = DB.http @@ -109,11 +133,10 @@ describe 'GraphDatabase::http', -> raw: true , _ - expect(resp).to.be.an.instanceOf http.IncomingMessage - expect(resp.statusCode).to.equal 405 # Method Not Allowed + expectResponse resp, 405 # Method Not Allowed it 'should throw native errors always', (_) -> - db = new GraphDatabase 'http://idontexist.foobarbaz.nodeneo4j' + db = new neo4j.GraphDatabase 'http://idontexist.foobarbaz.nodeneo4j' try thrown = false From 0133448a235c2a82a83144dcad967a1a0f01f97e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 23 Jan 2015 23:22:16 -0500 Subject: [PATCH 014/121] v2 / Errors: API, tests, partial impl. --- API_v2.md | 12 +++++++----- lib-new/GraphDatabase.coffee | 13 ++++++++----- lib-new/errors.coffee | 17 +++++++++++++++++ lib-new/index.coffee | 7 ++++++- test-new/core._coffee | 10 +++++++--- 5 files changed, 45 insertions(+), 14 deletions(-) create mode 100644 lib-new/errors.coffee diff --git a/API_v2.md b/API_v2.md index fc72d61..c7cc5fb 100644 --- a/API_v2.md +++ b/API_v2.md @@ -309,20 +309,22 @@ using `Error` subclasses. Importantly: - Special care is taken to provide `message` and `stack` properties rich in info, so that no special serialization is needed to debug production errors. -- And all info returned by Neo4j is also available on the `Error` instances +- Structured info returned by Neo4j is also available on the `Error` instances under a `neo4j` property, for deeper introspection and analysis if desired. + In addition, if this error is associated with a full HTTP response, the HTTP + `statusCode`, `headers`, and `body` are available via an `http` property. ```coffee -class Error {name, message, stack, neo4j} +class Error {name, message, stack, http, neo4j} class ClientError extends Error class DatabaseError extends Error class TransientError extends Error ``` -TODO: Should we name these classes with a `Neo4j` prefix? -They'll only be exposed via this driver's `module.exports`, so it's not -technically necessary, but that'd allow for e.g. destructuring. +Note that these class names do *not* have a `Neo4j` prefix, since they'll only +be exposed via this driver's `module.exports`, but for convenience, instances' +`name` properties *do* have a `neo4j.` prefix, e.g. `neo4j.ClientError`. ## Schema diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 10f76fa..f01be65 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,4 +1,5 @@ $ = require 'underscore' +errors = require './errors' lib = require '../package.json' Request = require 'request' URL = require 'url' @@ -45,7 +46,7 @@ module.exports = class GraphDatabase , (err, resp) => if err - # TODO: Properly check error and transform to semantic instance. + # TODO: Do we want to wrap or modify native errors? return cb err if raw @@ -55,13 +56,15 @@ module.exports = class GraphDatabase {body, headers, statusCode} = resp if statusCode >= 500 - # TODO: Semantic errors. - err = new Error "Neo4j #{statusCode} response" + # TODO: Parse errors, and differentiate w/ TransientErrors. + err = new errors.DatabaseError 'TODO', + http: {body, headers, statusCode} return cb err if statusCode >= 400 - # TODO: Semantic errors. - err = new Error "Neo4j #{statusCode} response" + # TODO: Parse errors. + err = new errors.ClientError 'TODO', + http: {body, headers, statusCode} return cb err # TODO: Parse nodes and relationships. diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee new file mode 100644 index 0000000..e4e75f6 --- /dev/null +++ b/lib-new/errors.coffee @@ -0,0 +1,17 @@ +$ = require 'underscore' + +class @Error extends Error + + Object.defineProperties @::, + name: {get: -> 'neo4j.' + @constructor.name} + + constructor: (@message='Unknown error', {@http, @neo4j}={}) -> + Error.captureStackTrace @, @constructor + + # TODO: Helper to rethrow native/inner errors? Not sure if we need one. + +class @ClientError extends @Error + +class @DatabaseError extends @Error + +class @TransientError extends @Error diff --git a/lib-new/index.coffee b/lib-new/index.coffee index b7f319c..edbcf2d 100644 --- a/lib-new/index.coffee +++ b/lib-new/index.coffee @@ -1 +1,6 @@ -exports.GraphDatabase = require './GraphDatabase' +$ = require 'underscore' + +$(exports).extend + GraphDatabase: require './GraphDatabase' + +$(exports).extend require './errors' diff --git a/test-new/core._coffee b/test-new/core._coffee index 29ede30..1364406 100644 --- a/test-new/core._coffee +++ b/test-new/core._coffee @@ -110,8 +110,10 @@ describe 'GraphDatabase::http', -> , _ catch err thrown = true - expect(err).to.be.an.instanceOf Error - # TODO: Deeper and more semantic assertions, e.g. status code. + expect(err).to.be.an.instanceOf neo4j.ClientError + expect(err.name).to.equal 'neo4j.ClientError' + expect(err.http).to.be.an 'object' + expect(err.http.statusCode).to.equal 405 expect(thrown).to.be.true() @@ -147,7 +149,9 @@ describe 'GraphDatabase::http', -> catch err thrown = true expect(err).to.be.an.instanceOf Error - # TODO: Deeper and more semantic assertions? + expect(err.name).to.equal 'Error' + expect(err.code).to.equal 'ENOTFOUND' + expect(err.syscall).to.equal 'getaddrinfo' expect(thrown).to.be.true() From f7b6239d7d81505bc8bc8149a58e3005e61104c1 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 27 Jan 2015 20:38:04 -0500 Subject: [PATCH 015/121] v2 / Tests: split up constructor and HTTP tests. --- test-new/constructor._coffee | 79 ++++++++++++++++++++++++ test-new/{core._coffee => http._coffee} | 81 +++++-------------------- 2 files changed, 94 insertions(+), 66 deletions(-) create mode 100644 test-new/constructor._coffee rename test-new/{core._coffee => http._coffee} (53%) diff --git a/test-new/constructor._coffee b/test-new/constructor._coffee new file mode 100644 index 0000000..34d4f4d --- /dev/null +++ b/test-new/constructor._coffee @@ -0,0 +1,79 @@ +# +# Tests for the GraphDatabase constructor, e.g. its options and overloads. +# + +$ = require 'underscore' +{expect} = require 'chai' +{GraphDatabase} = require '../' + + +## CONSTANTS + +URL = 'http://foo:bar@baz:1234' +PROXY = 'http://lorem.ipsum' +HEADERS = + 'x-foo': 'bar-baz' + 'x-lorem': 'ipsum' + # TODO: Test overlap with default headers? + # TODO: Test custom User-Agent behavior, or blacklist X-Stream? + + +## HELPERS + +# +# Asserts that the given object is an instance of GraphDatabase, +# pointing to the given URL, optionally using the given proxy URL. +# +expectDatabase = (db, url, proxy) -> + expect(db).to.be.an.instanceOf GraphDatabase + expect(db.url).to.equal url + expect(db.proxy).to.equal proxy + +# +# Asserts that the given GraphDatabase instance has its `headers` property set +# to the union of the given headers and the default GraphDatabase `headers`, +# with the given headers taking precedence. +# +# TODO: If we special-case User-Agent or blacklist X-Stream, update here. +# +expectHeaders = (db, headers) -> + expect(db.headers).to.be.an 'object' + + defaultHeaders = GraphDatabase::headers + defaultKeys = Object.keys defaultHeaders + givenKeys = Object.keys headers + expectedKeys = $.union defaultKeys, givenKeys # This de-dupes too. + + # This is an exact check, i.e. *only* these keys: + expect(db.headers).to.have.keys expectedKeys + + for key, val of db.headers + expect(val).to.equal headers[key] or defaultHeaders[key] + + +## TESTS + +describe 'GraphDatabase::constructor', -> + + it 'should support full options', -> + db = new GraphDatabase + url: URL + proxy: PROXY + headers: HEADERS + + expectDatabase db, URL, PROXY + expectHeaders db, HEADERS + + it 'should support just URL string', -> + db = new GraphDatabase URL + + expectDatabase db, URL + expectHeaders db, {} + + it 'should throw if no URL given', -> + fn = -> new GraphDatabase() + expect(fn).to.throw TypeError + + # Also try giving an options argument, just with no URL: + fn = -> new GraphDatabase {proxy: PROXY} + expect(fn).to.throw TypeError diff --git a/test-new/core._coffee b/test-new/http._coffee similarity index 53% rename from test-new/core._coffee rename to test-new/http._coffee index 1364406..66dcb9d 100644 --- a/test-new/core._coffee +++ b/test-new/http._coffee @@ -1,59 +1,32 @@ -$ = require 'underscore' +# +# Tests for the GraphDatabase `http` method, e.g. the ability to make custom, +# arbitrary HTTP requests, and have responses parsed for nodes, rels, & errors. +# + {expect} = require 'chai' http = require 'http' neo4j = require '../' -# SHARED STATE +## SHARED STATE -DB = null -URL = process.env.NEO4J_URL or 'http://localhost:7474' - -FAKE_PROXY = 'http://lorem.ipsum' -FAKE_HEADERS = - 'x-foo': 'bar-baz' - 'x-lorem': 'ipsum' - # TODO: Test overlap with default headers? - # TODO: Test custom User-Agent behavior, or blacklist X-Stream? +DB = new neo4j.GraphDatabase process.env.NEO4J_URL or 'http://localhost:7474' ## HELPERS # -# Asserts that the given object is an instance of GraphDatabase, -# pointing to the given URL, optionally using the given proxy URL. -# -expectDatabase = (db, url, proxy) -> - expect(db).to.be.an.instanceOf neo4j.GraphDatabase - expect(db.url).to.equal url - expect(db.proxy).to.equal proxy - +# Asserts that the given object is a proper HTTP client response with the given +# status code. # -# Asserts that the given GraphDatabase instance has its `headers` property set -# to the union of the given headers and the default GraphDatabase `headers`, -# with the given headers taking precedence. -# -# TODO: If we special-case User-Agent or blacklist X-Stream, update here. -# -expectHeaders = (db, headers) -> - expect(db.headers).to.be.an 'object' - - defaultHeaders = neo4j.GraphDatabase::headers - defaultKeys = Object.keys defaultHeaders - givenKeys = Object.keys headers - expectedKeys = $.union defaultKeys, givenKeys # This de-dupes too. - - # This is an exact check, i.e. *only* these keys: - expect(db.headers).to.have.keys expectedKeys - - for key, val of db.headers - expect(val).to.equal headers[key] or defaultHeaders[key] - expectResponse = (resp, statusCode) -> expect(resp).to.be.an.instanceOf http.IncomingMessage expect(resp.statusCode).to.equal statusCode expect(resp.headers).to.be.an 'object' +# +# Asserts that the given object is the root Neo4j object. +# expectNeo4jRoot = (body) -> expect(body).to.be.an 'object' expect(body).to.have.keys 'data', 'management' @@ -61,31 +34,6 @@ expectNeo4jRoot = (body) -> ## TESTS -describe 'GraphDatabase::constructor', -> - - it 'should support full options', -> - DB = new neo4j.GraphDatabase - url: URL - proxy: FAKE_PROXY - headers: FAKE_HEADERS - - expectDatabase DB, URL, FAKE_PROXY - expectHeaders DB, FAKE_HEADERS - - it 'should support just URL string', -> - DB = new neo4j.GraphDatabase URL - - expectDatabase DB, URL - expectHeaders DB, {} - - it 'should throw if no URL given', -> - fn = -> new neo4j.GraphDatabase() - expect(fn).to.throw TypeError - - # Also try giving an options argument, just with no URL: - fn = -> new neo4j.GraphDatabase {proxy: FAKE_PROXY} - expect(fn).to.throw TypeError - describe 'GraphDatabase::http', -> it 'should support simple GET requests by default', (_) -> @@ -96,7 +44,8 @@ describe 'GraphDatabase::http', -> body = DB.http method: 'GET' path: '/' - headers: FAKE_HEADERS + headers: + 'X-Foo': 'bar' , _ expectNeo4jRoot body @@ -155,7 +104,7 @@ describe 'GraphDatabase::http', -> expect(thrown).to.be.true() - it 'should support streaming' + it 'should support streaming (TODO)' # Test that it immediately returns a duplex HTTP stream. # Test writing request data to this stream. # Test reading response data from this stream. From 08b32d076d428eb9ba2f732045b1a9cb1e518ad2 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 29 Jan 2015 22:23:15 -0500 Subject: [PATCH 016/121] v2 / Core: impl. and test parsing of nodes & rels! --- lib-new/GraphDatabase.coffee | 41 +++++++++- lib-new/Node.coffee | 51 ++++++++++++ lib-new/Relationship.coffee | 44 +++++++++++ lib-new/index.coffee | 2 + lib-new/utils.coffee | 9 +++ test-new/fixtures/index._coffee | 13 +++ test-new/http._coffee | 135 +++++++++++++++++++++++++++++++- 7 files changed, 291 insertions(+), 4 deletions(-) create mode 100644 lib-new/Node.coffee create mode 100644 lib-new/Relationship.coffee create mode 100644 lib-new/utils.coffee create mode 100644 test-new/fixtures/index._coffee diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index f01be65..4d7e5af 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,6 +1,8 @@ $ = require 'underscore' errors = require './errors' lib = require '../package.json' +Node = require './Node' +Relationship = require './Relationship' Request = require 'request' URL = require 'url' @@ -36,7 +38,6 @@ module.exports = class GraphDatabase method or= 'GET' headers or= {} - # TODO: Parse response body before calling back. # TODO: Do we need to do anything special to support streaming response? req = Request method: method @@ -67,5 +68,39 @@ module.exports = class GraphDatabase http: {body, headers, statusCode} return cb err - # TODO: Parse nodes and relationships. - return cb null, body + # Parse nodes and relationships in the body, and return: + return cb null, _transform body + + +## HELPERS + +# +# Deep inspects the given object -- which could be a simple primitive, a map, +# an array with arbitrary other objects, etc. -- and transforms any objects that +# look like nodes and relationships into Node and Relationship instances. +# Returns the transformed object, and does not mutate the input object. +# +_transform = (obj) -> + # Nothing to transform for primitives and null: + if (not obj) or (typeof obj isnt 'object') + return obj + + # Process arrays: + # (NOTE: Not bothering to detect arrays made in other JS contexts.) + if obj instanceof Array + return obj.map _transform + + # Feature-detect (AKA "duck-type") Node & Relationship objects, by simply + # trying to parse them as such. + # Important: check relationships first, for precision/specificity. + # TODO: If we add a Path class, we'll need to check for that here too. + if rel = Relationship._fromRaw obj + return rel + if node = Node._fromRaw obj + return node + + # Otherwise, process as a dictionary/map: + map = {} + for key, val of obj + map[key] = _transform val + map diff --git a/lib-new/Node.coffee b/lib-new/Node.coffee new file mode 100644 index 0000000..7dc1e2e --- /dev/null +++ b/lib-new/Node.coffee @@ -0,0 +1,51 @@ +utils = require './utils' + +module.exports = class Node + + constructor: (opts={}) -> + {@_id, @labels, @properties} = opts + + equals: (other) -> + # TODO: Is this good enough? Often don't want exact equality, e.g. + # nodes' properties may change between queries. + (other instanceof Node) and (@_id is other._id) + + toString: -> + labels = @labels.map (label) -> ":#{label}" + "(#{@_id}#{labels.join ''})" # E.g. (123), (456:Foo), (789:Foo:Bar) + + # + # Accepts the given raw JSON from Neo4j's REST API, and if it represents a + # valid node, creates and returns a Node instance from it. + # If the JSON doesn't represent a valid node, returns null. + # + @_fromRaw: (obj) -> + return null if (not obj) or (typeof obj isnt 'object') + + {data, metadata, self} = obj + + return null if (not self) or (typeof self isnt 'string') or + (not data) or (typeof data isnt 'object') + + # Metadata was only added in Neo4j 2.1.5, so don't *require* it, + # but (a) it makes our job easier, and (b) it's the only way we can get + # labels, so warn the developer if it's missing, but only once. + if metadata + {id, labels} = metadata + else + id = utils.parseId self + labels = null + + if not @_warnedMetadata + @_warnedMetadata = true + console.warn 'It looks like you’re running Neo4j <2.1.5. + Neo4j <2.1.5 didn’t return label metadata to drivers, + so node-neo4j has no way to associate nodes with labels. + Thus, the `labels` property on node-neo4j `Node` instances + will always be null for you. Consider upgrading to fix. =) + http://neo4j.com/release-notes/neo4j-2-1-5/' + + return new Node + _id: id + labels: labels + properties: data diff --git a/lib-new/Relationship.coffee b/lib-new/Relationship.coffee new file mode 100644 index 0000000..fcd60e4 --- /dev/null +++ b/lib-new/Relationship.coffee @@ -0,0 +1,44 @@ +utils = require './utils' + +module.exports = class Relationship + + constructor: (opts={}) -> + {@_id, @type, @properties, @_fromId, @_toId} = opts + + equals: (other) -> + # TODO: Is this good enough? Often don't want exact equality, e.g. + # nodes' properties may change between queries. + (other instanceof Relationship) and (@_id is other._id) + + toString: -> + "-[#{@_id}:#{@type}]-" # E.g. -[123:FOLLOWS]- + + # + # Accepts the given raw JSON from Neo4j's REST API, and if it represents a + # valid relationship, creates and returns a Relationship instance from it. + # If the JSON doesn't represent a valid relationship, returns null. + # + @_fromRaw: (obj) -> + return null if (not obj) or (typeof obj isnt 'object') + + {data, self, type, start, end} = obj + + return null if (not self) or (typeof self isnt 'string') or + (not type) or (typeof type isnt 'string') or + (not start) or (typeof start isnt 'string') or + (not end) or (typeof end isnt 'string') or + (not data) or (typeof data isnt 'object') + + # Relationships also have `metadata`, added in Neo4j 2.1.5, but it + # doesn't provide anything new. (And it doesn't give us from/to ID.) + # We don't want to rely on it, so we don't bother using it at all. + id = utils.parseId self + fromId = utils.parseId start + toId = utils.parseId end + + return new Relationship + _id: id + type: type + properties: data + _fromId: fromId + _toId: toId diff --git a/lib-new/index.coffee b/lib-new/index.coffee index edbcf2d..150403b 100644 --- a/lib-new/index.coffee +++ b/lib-new/index.coffee @@ -2,5 +2,7 @@ $ = require 'underscore' $(exports).extend GraphDatabase: require './GraphDatabase' + Node: require './Node' + Relationship: require './Relationship' $(exports).extend require './errors' diff --git a/lib-new/utils.coffee b/lib-new/utils.coffee new file mode 100644 index 0000000..11485d9 --- /dev/null +++ b/lib-new/utils.coffee @@ -0,0 +1,9 @@ + +# +# Parses and returns the native Neo4j ID out of the given Neo4j URL, +# or null if no ID could be matched. +# +@parseId = (url) -> + match = url.match /// /db/data/(node|relationship)/(\d+)$ /// + return null if not match + return parseInt match[2], 10 diff --git a/test-new/fixtures/index._coffee b/test-new/fixtures/index._coffee new file mode 100644 index 0000000..b01f090 --- /dev/null +++ b/test-new/fixtures/index._coffee @@ -0,0 +1,13 @@ +# +# NOTE: This file is within a directory named `fixtures`, rather than a file +# named `fixtures._coffee`, in order to not have Mocha treat it like a test. +# + +neo4j = require '../../' + +exports.DB = + new neo4j.GraphDatabase process.env.NEO4J_URL or 'http://localhost:7474' + +exports.TEST_LABEL = 'Test' + +exports.TEST_REL_TYPE = 'TEST' diff --git a/test-new/http._coffee b/test-new/http._coffee index 66dcb9d..602172e 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -4,13 +4,29 @@ # {expect} = require 'chai' +fixtures = require './fixtures' http = require 'http' neo4j = require '../' ## SHARED STATE -DB = new neo4j.GraphDatabase process.env.NEO4J_URL or 'http://localhost:7474' +{DB, TEST_LABEL, TEST_REL_TYPE} = fixtures + +TEST_NODE_A = new neo4j.Node + # _id will get filled in once we persist + labels: [TEST_LABEL] + properties: {suite: module.filename, name: 'a'} + +TEST_NODE_B = new neo4j.Node + # _id will get filled in once we persist + labels: [TEST_LABEL] + properties: {suite: module.filename, name: 'b'} + +TEST_REL = new neo4j.Relationship + # _id, _fromId (node A), _toId (node B) will get filled in once we persist + type: TEST_REL_TYPE + properties: {suite: module.filename, name: 'r'} ## HELPERS @@ -108,3 +124,120 @@ describe 'GraphDatabase::http', -> # Test that it immediately returns a duplex HTTP stream. # Test writing request data to this stream. # Test reading response data from this stream. + + + ## Object parsing: + + it '(create test objects)', (_) -> + # NOTE: Using the old Cypher endpoint for simplicity here. + # Nicer than using the raw REST API to create these test objects, + # but also nice to neither use this driver's Cypher functionality + # (which is tested in a higher-level test suite), nor re-implement it. + # http://neo4j.com/docs/stable/rest-api-cypher.html#rest-api-use-parameters + {data} = DB.http + method: 'POST' + path: '/db/data/cypher' + body: + query: """ + CREATE (a:#{TEST_LABEL} {propsA}) + CREATE (b:#{TEST_LABEL} {propsB}) + CREATE (a) -[r:#{TEST_REL_TYPE} {propsR}]-> (b) + RETURN ID(a), ID(b), ID(r) + """ + params: + propsA: TEST_NODE_A.properties + propsB: TEST_NODE_B.properties + propsR: TEST_REL.properties + , _ + + [row] = data + [idA, idB, idR] = row + + TEST_NODE_A._id = idA + TEST_NODE_B._id = idB + TEST_REL._id = idR + TEST_REL._fromId = idA + TEST_REL._toId = idB + + it 'should parse nodes by default', (_) -> + node = DB.http + method: 'GET' + path: "/db/data/node/#{TEST_NODE_A._id}" + , _ + + expect(node).to.be.an.instanceOf neo4j.Node + expect(node).to.eql TEST_NODE_A + + it 'should parse relationships by default', (_) -> + rel = DB.http + method: 'GET' + path: "/db/data/relationship/#{TEST_REL._id}" + , _ + + expect(rel).to.be.an.instanceOf neo4j.Relationship + expect(rel).to.eql TEST_REL + + it 'should parse nested nodes & relationships by default', (_) -> + {data} = DB.http + method: 'POST' + path: '/db/data/cypher' + body: + query: """ + START a = node({idA}) + MATCH (a) -[r]-> (b) + RETURN a, r, b + """ + params: + idA: TEST_NODE_A._id + , _ + + [row] = data + [nodeA, rel, nodeB] = row + + expect(nodeA).to.be.an.instanceOf neo4j.Node + expect(nodeA).to.eql TEST_NODE_A + expect(nodeB).to.be.an.instanceOf neo4j.Node + expect(nodeB).to.eql TEST_NODE_B + expect(rel).to.be.an.instanceOf neo4j.Relationship + expect(rel).to.eql TEST_REL + + it 'should not parse nodes for raw responses', (_) -> + {body} = DB.http + method: 'GET' + path: "/db/data/node/#{TEST_NODE_A._id}" + raw: true + , _ + + expect(body).to.not.be.an.instanceOf neo4j.Node + expect(body.metadata).to.be.an 'object' + expect(body.metadata.id).to.equal TEST_NODE_A._id + expect(body.metadata.labels).to.eql TEST_NODE_A.labels + expect(body.data).to.eql TEST_NODE_A.properties + + it 'should not parse relationships for raw responses', (_) -> + {body} = DB.http + method: 'GET' + path: "/db/data/relationship/#{TEST_REL._id}" + raw: true + , _ + + expect(body.metadata).to.be.an 'object' + expect(body.metadata.id).to.equal TEST_REL._id + expect(body).to.not.be.an.instanceOf neo4j.Relationship + expect(body.type).to.equal TEST_REL.type + expect(body.data).to.eql TEST_REL.properties + + it '(delete test objects)', (_) -> + DB.http + method: 'POST' + path: '/db/data/cypher' + body: + query: """ + START a = node({idA}), b = node({idB}), r = rel({idR}) + DELETE a, b, r + """ + params: + idA: TEST_NODE_A._id + idB: TEST_NODE_B._id + idR: TEST_REL._id + , _ From 5d76c876ec172883eb92e6badfad85840875b176 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 30 Jan 2015 01:16:03 -0500 Subject: [PATCH 017/121] v2 / Errors: improve impl. and tests. --- API_v2.md | 6 +- lib-new/GraphDatabase.coffee | 16 +---- lib-new/errors.coffee | 38 ++++++++++-- test-new/http._coffee | 116 +++++++++++++++++++++++++---------- 4 files changed, 125 insertions(+), 51 deletions(-) diff --git a/API_v2.md b/API_v2.md index c7cc5fb..2cfb4fb 100644 --- a/API_v2.md +++ b/API_v2.md @@ -308,14 +308,14 @@ using `Error` subclasses. Importantly: - Special care is taken to provide `message` and `stack` properties rich in info, so that no special serialization is needed to debug production errors. + That is, simply logging the `stack` (as most error handlers tend to do) + should get you meaningful and useful data. - Structured info returned by Neo4j is also available on the `Error` instances under a `neo4j` property, for deeper introspection and analysis if desired. - In addition, if this error is associated with a full HTTP response, the HTTP - `statusCode`, `headers`, and `body` are available via an `http` property. ```coffee -class Error {name, message, stack, http, neo4j} +class Error {name, message, stack, neo4j} class ClientError extends Error class DatabaseError extends Error diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 4d7e5af..22deab4 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,5 +1,5 @@ $ = require 'underscore' -errors = require './errors' +{Error} = require './errors' lib = require '../package.json' Node = require './Node' Relationship = require './Relationship' @@ -56,20 +56,10 @@ module.exports = class GraphDatabase {body, headers, statusCode} = resp - if statusCode >= 500 - # TODO: Parse errors, and differentiate w/ TransientErrors. - err = new errors.DatabaseError 'TODO', - http: {body, headers, statusCode} + if err = Error._fromResponse resp return cb err - if statusCode >= 400 - # TODO: Parse errors. - err = new errors.ClientError 'TODO', - http: {body, headers, statusCode} - return cb err - - # Parse nodes and relationships in the body, and return: - return cb null, _transform body + cb null, _transform body ## HELPERS diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index e4e75f6..e53eb9a 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -1,13 +1,43 @@ $ = require 'underscore' +http = require 'http' class @Error extends Error - Object.defineProperties @::, - name: {get: -> 'neo4j.' + @constructor.name} - - constructor: (@message='Unknown error', {@http, @neo4j}={}) -> + constructor: (@message='Unknown error', @neo4j={}) -> + @name = 'neo4j.' + @constructor.name Error.captureStackTrace @, @constructor + # + # Accepts the given HTTP client response, and if it represents an error, + # creates and returns the appropriate Error instance from it. + # If the response doesn't represent an error, returns null. + # + @_fromResponse: (resp) -> + {body, headers, statusCode} = resp + + return null if statusCode < 400 + + # TODO: Do some status codes (or perhaps inner `exception` names) + # signify Transient errors rather than Database ones? + ErrorType = if statusCode >= 500 then 'Database' else 'Client' + ErrorClass = exports["#{ErrorType}Error"] + + message = "[#{statusCode}] " + logBody = statusCode >= 500 # TODO: Config to always log body? + + if body?.exception + message += "[#{body.exception}] #{body.message or '(no message)'}" + else + statusText = http.STATUS_CODES[statusCode] # E.g. "Not Found" + reqText = "#{resp.req.method} #{resp.req.path}" + message += "#{statusText} response for #{reqText}" + logBody = true # always log body if non-error returned + + if logBody and body? + message += ": #{JSON.stringify body, null, 4}" + + new ErrorClass message, body + # TODO: Helper to rethrow native/inner errors? Not sure if we need one. class @ClientError extends @Error diff --git a/test-new/http._coffee b/test-new/http._coffee index 602172e..4af50fb 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -47,6 +47,19 @@ expectNeo4jRoot = (body) -> expect(body).to.be.an 'object' expect(body).to.have.keys 'data', 'management' +# +# Asserts that the given object is a proper instance of the given Neo4j Error +# subclass, including with the given message. +# Additional checks, e.g. of the `neo4j` property's contents, are up to you. +# +expectError = (err, ErrorClass, message) -> + expect(err).to.be.an.instanceOf ErrorClass + expect(err.name).to.equal "neo4j.#{ErrorClass.name}" + expect(err.neo4j).to.be.an 'object' + expect(err.message).to.equal message + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + ## TESTS @@ -66,21 +79,53 @@ describe 'GraphDatabase::http', -> expectNeo4jRoot body - it 'should throw errors for 4xx responses by default', (_) -> - try - thrown = false - DB.http - method: 'POST' - path: '/' - , _ - catch err - thrown = true - expect(err).to.be.an.instanceOf neo4j.ClientError - expect(err.name).to.equal 'neo4j.ClientError' - expect(err.http).to.be.an 'object' - expect(err.http.statusCode).to.equal 405 - - expect(thrown).to.be.true() + it 'should throw errors for 4xx responses by default', (done) -> + DB.http + method: 'POST' + path: '/' + , (err, body) -> + try + expect(err).to.exist() + expect(body).to.not.exist() + + expectError err, neo4j.ClientError, + '[405] Method Not Allowed response for POST /' + expect(err.neo4j).to.be.empty() + + catch assertionErr + return done assertionErr + + done() + + it 'should properly parse Neo4j exceptions', (done) -> + DB.http + method: 'GET' + path: '/db/data/node/-1' + , (err, body) -> + try + expect(err).to.exist() + expect(body).to.not.exist() + + expectError err, neo4j.ClientError, '[404] [NodeNotFoundException] + Cannot find node with id [-1] in database.' + + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.exception).to.equal 'NodeNotFoundException' + expect(err.neo4j.fullname).to.equal ' + org.neo4j.server.rest.web.NodeNotFoundException' + expect(err.neo4j.message).to.equal ' + Cannot find node with id [-1] in database.' + + expect(err.neo4j.stacktrace).to.be.an 'array' + expect(err.neo4j.stacktrace).to.not.be.empty() + for line in err.neo4j.stacktrace + expect(line).to.be.a 'string' + expect(line).to.not.be.empty() + + catch assertionErr + return done assertionErr + + done() it 'should support returning raw responses', (_) -> resp = DB.http @@ -102,23 +147,32 @@ describe 'GraphDatabase::http', -> expectResponse resp, 405 # Method Not Allowed - it 'should throw native errors always', (_) -> + it 'should throw native errors always', (done) -> db = new neo4j.GraphDatabase 'http://idontexist.foobarbaz.nodeneo4j' - - try - thrown = false - db.http - path: '/' - raw: true - , _ - catch err - thrown = true - expect(err).to.be.an.instanceOf Error - expect(err.name).to.equal 'Error' - expect(err.code).to.equal 'ENOTFOUND' - expect(err.syscall).to.equal 'getaddrinfo' - - expect(thrown).to.be.true() + db.http + path: '/' + raw: true + , (err, resp) -> + try + expect(err).to.exist() + expect(resp).to.not.exist() + + # NOTE: *Not* using `expectError` here, because we explicitly + # don't wrap native (non-Neo4j) errors. + expect(err).to.be.an.instanceOf Error + expect(err.name).to.equal 'Error' + expect(err.code).to.equal 'ENOTFOUND' + expect(err.syscall).to.equal 'getaddrinfo' + expect(err.message).to.contain "#{err.syscall} #{err.code}" + # NOTE: Node 0.12 adds the hostname to the message. + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal \ + "#{err.name}: #{err.message}" + + catch assertionErr + return done assertionErr + + done() it 'should support streaming (TODO)' # Test that it immediately returns a duplex HTTP stream. From c47f52c67b0197e626d261392168af95014846da Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 30 Jan 2015 02:05:35 -0500 Subject: [PATCH 018/121] v2 / Tests: update Travis; fix for older Neo4j versions. --- .travis.yml | 13 ++++++++++--- test-new/http._coffee | 39 ++++++++++++++++++++++++++++++++++----- 2 files changed, 44 insertions(+), 8 deletions(-) diff --git a/.travis.yml b/.travis.yml index 4f84734..d079e2a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,7 +1,7 @@ language: node_js node_js: - - "0.11" + - "0.12" - "0.10" jdk: @@ -9,9 +9,16 @@ jdk: env: # test across multiple versions of Neo4j: - - NEO4J_VERSION="2.1.5" + - NEO4J_VERSION="2.2.0-M03" + - NEO4J_VERSION="2.1.6" - NEO4J_VERSION="2.0.4" - - NEO4J_VERSION="1.9.8" + +matrix: + # but we may want to allow our tests to fail against future, unstable + # versions of Neo4j. E.g. 2.2 introduces auth, and sets it by default. + # TODO: remove this once we've added auth support and fixed it for 2.2. + allow_failures: + - env: NEO4J_VERSION="2.2.0-M03" before_install: # install Neo4j locally: diff --git a/test-new/http._coffee b/test-new/http._coffee index 4af50fb..6bb6162 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -13,6 +13,8 @@ neo4j = require '../' {DB, TEST_LABEL, TEST_REL_TYPE} = fixtures +[DB_VERSION_STR, DB_VERSION_NUM] = [] + TEST_NODE_A = new neo4j.Node # _id will get filled in once we persist labels: [TEST_LABEL] @@ -182,6 +184,25 @@ describe 'GraphDatabase::http', -> ## Object parsing: + it '(query Neo4j version)', (_) -> + info = DB.http + method: 'GET' + path: '/db/data/' + , _ + + DB_VERSION_STR = info.neo4j_version or '0' + DB_VERSION_NUM = parseFloat DB_VERSION_STR, 10 + + if DB_VERSION_NUM < 2 + throw new Error '*** node-neo4j v2 supports Neo4j v2+ only, + and you’re running Neo4j v1. These tests will fail! ***' + + # Neo4j <2.1.5 didn't return label info, so returned nodes won't have + # the labels we expect. Account for that: + if DB_VERSION_STR < '2.1.5' + TEST_NODE_A.labels = null + TEST_NODE_B.labels = null + it '(create test objects)', (_) -> # NOTE: Using the old Cypher endpoint for simplicity here. # Nicer than using the raw REST API to create these test objects, @@ -263,9 +284,13 @@ describe 'GraphDatabase::http', -> , _ expect(body).to.not.be.an.instanceOf neo4j.Node - expect(body.metadata).to.be.an 'object' - expect(body.metadata.id).to.equal TEST_NODE_A._id - expect(body.metadata.labels).to.eql TEST_NODE_A.labels + + # NOTE: Neo4j <2.1.5 didn't return `metadata`, so can't rely on it: + if DB_VERSION_STR >= '2.1.5' + expect(body.metadata).to.be.an 'object' + expect(body.metadata.id).to.equal TEST_NODE_A._id + expect(body.metadata.labels).to.eql TEST_NODE_A.labels + expect(body.data).to.eql TEST_NODE_A.properties it 'should not parse relationships for raw responses', (_) -> @@ -275,9 +300,13 @@ describe 'GraphDatabase::http', -> raw: true , _ - expect(body.metadata).to.be.an 'object' - expect(body.metadata.id).to.equal TEST_REL._id expect(body).to.not.be.an.instanceOf neo4j.Relationship + + # NOTE: Neo4j <2.1.5 didn't return `metadata`, so can't rely on it: + if DB_VERSION_STR >= '2.1.5' + expect(body.metadata).to.be.an 'object' + expect(body.metadata.id).to.equal TEST_REL._id + expect(body.type).to.equal TEST_REL.type expect(body.data).to.eql TEST_REL.properties From f122645a9bcca2fde0ffbb39584973d652249fd5 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 30 Jan 2015 20:13:36 -0500 Subject: [PATCH 019/121] v2 / Cypher: impl. and test single queries! --- API_v2.md | 3 + lib-new/GraphDatabase.coffee | 67 +++++++++ lib-new/errors.coffee | 20 +++ lib-new/utils.coffee | 11 +- test-new/cypher._coffee | 254 +++++++++++++++++++++++++++++++++++ 5 files changed, 351 insertions(+), 4 deletions(-) create mode 100644 test-new/cypher._coffee diff --git a/API_v2.md b/API_v2.md index 2cfb4fb..0dbc952 100644 --- a/API_v2.md +++ b/API_v2.md @@ -147,6 +147,9 @@ from database responses. function cb(err, results) {}; var stream = db.cypher({query, params, headers, raw}, cb); + +// Alternate simple version -- no params, no headers, not raw: +var stream = db.cypher(query, cb); ``` If a callback is given, it'll be called with the array of results, diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 22deab4..2e0afad 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -5,6 +5,8 @@ Node = require './Node' Relationship = require './Relationship' Request = require 'request' URL = require 'url' +utils = require './utils' + module.exports = class GraphDatabase @@ -61,6 +63,71 @@ module.exports = class GraphDatabase cb null, _transform body + cypher: (opts={}, cb) -> + if typeof opts is 'string' + opts = {query: opts} + + {query, params, headers, raw} = opts + + method = 'POST' + path = '/db/data/transaction/commit' + + # NOTE: Lowercase 'rest' matters here for parsing. + format = if raw then 'row' else 'rest' + statements = [] + body = {statements} + + # TODO: Support batching multiple queries in this request? + if query + statements.push + statement: query + parameters: params or {} + resultDataContents: [format] + + # TODO: Support streaming! + @http {method, path, headers, body}, (err, body) => + + if err + # TODO: Do we want to wrap or modify native errors? + # NOTE: This includes our own errors for non-2xx responses. + return cb err + + {results, errors} = body + + if errors.length + # TODO: Is it possible to get back more than one error? + # If so, is it fine for us to just use the first one? + [error] = errors + return cb Error._fromTransaction error + + # If there are no results, it means no statements were sent + # (e.g. to commit, rollback, or renew a transaction in isolation), + # so nothing to return, i.e. a void call in that case. + # Important: we explicitly don't return an empty array, because that + # implies we *did* send a query, that just didn't match anything. + if not results.length + return cb null, null + + # The top-level `results` is an array of results corresponding to + # the `statements` (queries) inputted. + # We send only one statement/query, so we have only one result. + [result] = results + {columns, data} = result + + # The `data` is an array of result rows, but each of its elements is + # actually a dictionary of results keyed by *response format*. + # We only request one format, `rest` by default, `row` if `raw`. + # In both cases, the value is an array of rows, where each row is an + # array of column values. + # We transform those rows into dictionaries keyed by column names. + results = $(data).pluck(format).map (row) -> + result = {} + for column, i in columns + result[column] = row[i] + result + + cb null, results + ## HELPERS diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index e53eb9a..3991e98 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -38,6 +38,26 @@ class @Error extends Error new ErrorClass message, body + # + # Accepts the given error object from a transactional Cypher response, and + # creates and returns the appropriate Error instance for it. + # + @_fromTransaction: (obj) -> + # http://neo4j.com/docs/stable/rest-api-transactional.html#rest-api-handling-errors + # http://neo4j.com/docs/stable/status-codes.html + {code, message} = obj + [neo, classification, category, title] = code.split '.' + + ErrorClass = exports[classification] # e.g. DatabaseError + message = "[#{category}.#{title}] #{message or '(no message)'}" + + # TODO: Some errors (always DatabaseErrors?) can also apparently have a + # `stack` property with the Java stack trace. Should we include it in + # our own message/stack, in the DatabaseError case at least? + # (This'd be analagous to including the body for 5xx responses above.) + + new ErrorClass message, obj + # TODO: Helper to rethrow native/inner errors? Not sure if we need one. class @ClientError extends @Error diff --git a/lib-new/utils.coffee b/lib-new/utils.coffee index 11485d9..ed75112 100644 --- a/lib-new/utils.coffee +++ b/lib-new/utils.coffee @@ -1,9 +1,12 @@ # -# Parses and returns the native Neo4j ID out of the given Neo4j URL, -# or null if no ID could be matched. +# Parses and returns the top-level native Neo4j ID out of the given Neo4j URL, +# or null if no top-level ID could be matched. +# +# Works with any type of object exposed by Neo4j at the root of the service API, +# e.g. nodes, relationships, even transactions. # @parseId = (url) -> - match = url.match /// /db/data/(node|relationship)/(\d+)$ /// + match = url.match /// /db/data/\w+/(\d+)($|/) /// return null if not match - return parseInt match[2], 10 + return parseInt match[1], 10 diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee new file mode 100644 index 0000000..623be05 --- /dev/null +++ b/test-new/cypher._coffee @@ -0,0 +1,254 @@ +# +# Tests for the GraphDatabase `cypher` method, e.g. the ability to make queries, +# parametrize them, have responses be auto-parsed for nodes & rels, etc. +# + +{expect} = require 'chai' +fixtures = require './fixtures' +neo4j = require '../' + + +## SHARED STATE +# TODO: De-dupe these with the HTTP test suite. + +{DB, TEST_LABEL, TEST_REL_TYPE} = fixtures + +[DB_VERSION_STR, DB_VERSION_NUM] = [] + +TEST_NODE_A = new neo4j.Node + # _id will get filled in once we persist + labels: [TEST_LABEL] + properties: {suite: module.filename, name: 'a'} + +TEST_NODE_B = new neo4j.Node + # _id will get filled in once we persist + labels: [TEST_LABEL] + properties: {suite: module.filename, name: 'b'} + +TEST_REL = new neo4j.Relationship + # _id, _fromId (node A), _toId (node B) will get filled in once we persist + type: TEST_REL_TYPE + properties: {suite: module.filename, name: 'r'} + + +## HELPERS + +# +# Asserts that the given object is an instance of the proper Neo4j Error +# subclass, representing the given transactional Neo4j error info. +# TODO: Consider consolidating with a similar helper in the `http` test suite. +# +expectError = (err, classification, category, title, message) -> + expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError + expect(err.name).to.equal "neo4j.#{classification}" + expect(err.message).to.equal "[#{category}.#{title}] #{message}" + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j).to.eql + code: "Neo.#{classification}.#{category}.#{title}" + message: message + + +## TESTS + +describe 'GraphDatabase::cypher', -> + + it 'should support simple queries and results', (_) -> + results = DB.cypher 'RETURN "bar" AS foo', _ + + expect(results).to.be.an 'array' + expect(results).to.have.length 1 + + [result] = results + + expect(result).to.be.an 'object' + expect(result).to.contain.keys 'foo' # this is an exact/"only" check + expect(result.foo).to.equal 'bar' + + it 'should support simple parameters', (_) -> + [result] = DB.cypher + query: 'RETURN {foo} AS foo' + params: {foo: 'bar'} + , _ + + expect(result).to.be.an 'object' + expect(result.foo).to.equal 'bar' + + it 'should support complex queries, params, and results', (_) -> + results = DB.cypher + query: ''' + UNWIND {nodes} AS node + WITH node + ORDER BY node.id + LIMIT {count} + RETURN node.id, node.name, {count} + ''' + params: + count: 3 + nodes: [ + {id: 2, name: 'Bob'} + {id: 4, name: 'Dave'} + {id: 1, name: 'Alice'} + {id: 3, name: 'Carol'} + ] + , _ + + expect(results).to.eql [ + 'node.id': 1 + 'node.name': 'Alice' + '{count}': 3 + , + 'node.id': 2 + 'node.name': 'Bob' + '{count}': 3 + , + 'node.id': 3 + 'node.name': 'Carol' + '{count}': 3 + ] + + it 'should support queries that return nothing', (_) -> + results = DB.cypher + query: 'MATCH (n:FooBarBazThisLabelDoesntExist) RETURN n' + params: {unused: 'param'} + , _ + + expect(results).to.be.empty() + + it 'should reject empty/missing queries', -> + fn1 = -> DB.cypher '', -> + fn2 = -> DB.cypher {}, -> + expect(fn1).to.throw TypeError, /query/i + expect(fn2).to.throw TypeError, /query/i + + it 'should properly parse and throw Neo4j errors', (done) -> + DB.cypher 'RETURN {foo}', (err, results) -> + try + expect(err).to.exist() + expect(results).to.not.exist() + + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + + catch assertionErr + return done assertionErr + + done() + + # TODO: De-dupe with the HTTP test suite. + it '(query Neo4j version)', (_) -> + info = DB.http + method: 'GET' + path: '/db/data/' + , _ + + DB_VERSION_STR = info.neo4j_version or '0' + DB_VERSION_NUM = parseFloat DB_VERSION_STR, 10 + + if DB_VERSION_NUM < 2 + throw new Error '*** node-neo4j v2 supports Neo4j v2+ only, + and you’re running Neo4j v1. These tests will fail! ***' + + # Neo4j <2.1.5 didn't return label info, so returned nodes won't have + # the labels we expect. Account for that: + if DB_VERSION_STR < '2.1.5' + TEST_NODE_A.labels = null + TEST_NODE_B.labels = null + + it 'should properly parse nodes & relationships', (_) -> + # We do a complex return to test nested/wrapped objects too. + # NOTE: However, returning an array changes the order of the returned + # results, no longer the deterministic order of [a, b, r]. + # We overcome this by explicitly indexing and ordering. + results = DB.cypher + query: """ + CREATE (a:#{TEST_LABEL} {propsA}) + CREATE (b:#{TEST_LABEL} {propsB}) + CREATE (a) -[r:#{TEST_REL_TYPE} {propsR}]-> (b) + WITH [ + {i: 0, elmt: a, id: ID(a)}, + {i: 1, elmt: b, id: ID(b)}, + {i: 2, elmt: r, id: ID(r)} + ] AS array + UNWIND array AS obj + RETURN obj.i, obj.id AS _id, [{ + inner: obj.elmt + }] AS outer + ORDER BY obj.i + """ + params: + propsA: TEST_NODE_A.properties + propsB: TEST_NODE_B.properties + propsR: TEST_REL.properties + , _ + + # We need to grab the native IDs of the objects we created, but after + # that, we can just compare object equality for simplicity. + + expect(results).to.have.length 3 + + [resultA, resultB, resultR] = results + + TEST_NODE_A._id = resultA._id + TEST_NODE_B._id = resultB._id + TEST_REL._id = resultR._id + TEST_REL._fromId = TEST_NODE_A._id + TEST_REL._toId = TEST_NODE_B._id + + expect(results).to.eql [ + 'obj.i': 0 + _id: TEST_NODE_A._id + outer: [ + inner: TEST_NODE_A + ] + , + 'obj.i': 1 + _id: TEST_NODE_B._id + outer: [ + inner: TEST_NODE_B + ] + , + 'obj.i': 2 + _id: TEST_REL._id + outer: [ + inner: TEST_REL + ] + ] + + # But also test that the returned objects are proper instances: + expect(resultA.outer[0].inner).to.be.an.instanceOf neo4j.Node + expect(resultB.outer[0].inner).to.be.an.instanceOf neo4j.Node + expect(resultR.outer[0].inner).to.be.an.instanceOf neo4j.Relationship + + it 'should not parse nodes & relationships if raw', (_) -> + results = DB.cypher + query: """ + START a = node({idA}) + MATCH (a) -[r]-> (b) + RETURN a, b, r + """ + params: + idA: TEST_NODE_A._id + raw: true + , _ + + expect(results).to.eql [ + a: TEST_NODE_A.properties + b: TEST_NODE_B.properties + r: TEST_REL.properties + ] + + it 'should support streaming (TODO)' + + it '(delete test objects)', (_) -> + DB.cypher + query: """ + START a = node({idA}), b = node({idB}), r = rel({idR}) + DELETE a, b, r + """ + params: + idA: TEST_NODE_A._id + idB: TEST_NODE_B._id + idR: TEST_REL._id + , _ From ce97680d8a8b4ad0a49b256ec0c150623461392d Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 3 Feb 2015 13:24:24 -0500 Subject: [PATCH 020/121] v2 / Tests: de-dupe test graph queries and logic. --- test-new/cypher._coffee | 102 ++++--------------- test-new/fixtures/index._coffee | 168 +++++++++++++++++++++++++++++++- test-new/http._coffee | 91 ++--------------- 3 files changed, 195 insertions(+), 166 deletions(-) diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 623be05..9e976d6 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -9,26 +9,10 @@ neo4j = require '../' ## SHARED STATE -# TODO: De-dupe these with the HTTP test suite. -{DB, TEST_LABEL, TEST_REL_TYPE} = fixtures +{DB} = fixtures -[DB_VERSION_STR, DB_VERSION_NUM] = [] - -TEST_NODE_A = new neo4j.Node - # _id will get filled in once we persist - labels: [TEST_LABEL] - properties: {suite: module.filename, name: 'a'} - -TEST_NODE_B = new neo4j.Node - # _id will get filled in once we persist - labels: [TEST_LABEL] - properties: {suite: module.filename, name: 'b'} - -TEST_REL = new neo4j.Relationship - # _id, _fromId (node A), _toId (node B) will get filled in once we persist - type: TEST_REL_TYPE - properties: {suite: module.filename, name: 'r'} +[TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] ## HELPERS @@ -136,25 +120,9 @@ describe 'GraphDatabase::cypher', -> done() - # TODO: De-dupe with the HTTP test suite. - it '(query Neo4j version)', (_) -> - info = DB.http - method: 'GET' - path: '/db/data/' - , _ - - DB_VERSION_STR = info.neo4j_version or '0' - DB_VERSION_NUM = parseFloat DB_VERSION_STR, 10 - - if DB_VERSION_NUM < 2 - throw new Error '*** node-neo4j v2 supports Neo4j v2+ only, - and you’re running Neo4j v1. These tests will fail! ***' - - # Neo4j <2.1.5 didn't return label info, so returned nodes won't have - # the labels we expect. Account for that: - if DB_VERSION_STR < '2.1.5' - TEST_NODE_A.labels = null - TEST_NODE_B.labels = null + it '(create test graph)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ it 'should properly parse nodes & relationships', (_) -> # We do a complex return to test nested/wrapped objects too. @@ -163,63 +131,44 @@ describe 'GraphDatabase::cypher', -> # We overcome this by explicitly indexing and ordering. results = DB.cypher query: """ - CREATE (a:#{TEST_LABEL} {propsA}) - CREATE (b:#{TEST_LABEL} {propsB}) - CREATE (a) -[r:#{TEST_REL_TYPE} {propsR}]-> (b) + START a = node({idA}) + MATCH (a) -[r]-> (b) WITH [ - {i: 0, elmt: a, id: ID(a)}, - {i: 1, elmt: b, id: ID(b)}, - {i: 2, elmt: r, id: ID(r)} + {i: 0, elmt: a}, + {i: 1, elmt: b}, + {i: 2, elmt: r} ] AS array UNWIND array AS obj - RETURN obj.i, obj.id AS _id, [{ + RETURN obj.i AS i, [{ inner: obj.elmt }] AS outer - ORDER BY obj.i + ORDER BY i """ params: - propsA: TEST_NODE_A.properties - propsB: TEST_NODE_B.properties - propsR: TEST_REL.properties + idA: TEST_NODE_A._id , _ - # We need to grab the native IDs of the objects we created, but after - # that, we can just compare object equality for simplicity. - - expect(results).to.have.length 3 - - [resultA, resultB, resultR] = results - - TEST_NODE_A._id = resultA._id - TEST_NODE_B._id = resultB._id - TEST_REL._id = resultR._id - TEST_REL._fromId = TEST_NODE_A._id - TEST_REL._toId = TEST_NODE_B._id - expect(results).to.eql [ - 'obj.i': 0 - _id: TEST_NODE_A._id + i: 0 outer: [ inner: TEST_NODE_A ] , - 'obj.i': 1 - _id: TEST_NODE_B._id + i: 1 outer: [ inner: TEST_NODE_B ] , - 'obj.i': 2 - _id: TEST_REL._id + i: 2 outer: [ inner: TEST_REL ] ] # But also test that the returned objects are proper instances: - expect(resultA.outer[0].inner).to.be.an.instanceOf neo4j.Node - expect(resultB.outer[0].inner).to.be.an.instanceOf neo4j.Node - expect(resultR.outer[0].inner).to.be.an.instanceOf neo4j.Relationship + expect(results[0].outer[0].inner).to.be.an.instanceOf neo4j.Node + expect(results[1].outer[0].inner).to.be.an.instanceOf neo4j.Node + expect(results[2].outer[0].inner).to.be.an.instanceOf neo4j.Relationship it 'should not parse nodes & relationships if raw', (_) -> results = DB.cypher @@ -241,14 +190,5 @@ describe 'GraphDatabase::cypher', -> it 'should support streaming (TODO)' - it '(delete test objects)', (_) -> - DB.cypher - query: """ - START a = node({idA}), b = node({idB}), r = rel({idR}) - DELETE a, b, r - """ - params: - idA: TEST_NODE_A._id - idB: TEST_NODE_B._id - idR: TEST_REL._id - , _ + it '(delete test graph)', (_) -> + fixtures.deleteTestGraph module, _ diff --git a/test-new/fixtures/index._coffee b/test-new/fixtures/index._coffee index b01f090..5057ed0 100644 --- a/test-new/fixtures/index._coffee +++ b/test-new/fixtures/index._coffee @@ -3,11 +3,173 @@ # named `fixtures._coffee`, in order to not have Mocha treat it like a test. # +$ = require 'underscore' +{expect} = require 'chai' neo4j = require '../../' -exports.DB = +@DB = new neo4j.GraphDatabase process.env.NEO4J_URL or 'http://localhost:7474' -exports.TEST_LABEL = 'Test' +# We fill these in, and cache them, the first time tests request them: +@DB_VERSION_NUM = null +@DB_VERSION_STR = null -exports.TEST_REL_TYPE = 'TEST' +@TEST_LABEL = 'Test' +@TEST_REL_TYPE = 'TEST' + +# +# Queries the Neo4j version of the database we're currently testing against, +# if it's not already known. +# Doesn't return anything; instead, @DB_VERSION_* will be set after this. +# +@queryDbVersion = (_) => + return if @DB_VERSION_NUM + + info = @DB.http + method: 'GET' + path: '/db/data/' + , _ + + @DB_VERSION_STR = info.neo4j_version or '(version unknown)' + @DB_VERSION_NUM = parseFloat @DB_VERSION_STR, 10 + + if @DB_VERSION_NUM < 2 + throw new Error "*** node-neo4j v2 supports Neo4j v2+ only, + and you’re running Neo4j v1. These tests will fail! ***" + +# +# Returns a random string. +# +@getRandomStr = -> + "#{Math.random()}"[2..] + +# +# Creates and returns a property bag (dictionary) with unique, random test data +# for the given test suite (pass the suite's Node `module`). +# +@createTestProperties = (suite) => + suite: suite.filename + rand: @getRandomStr() + +# +# Creates and returns a new Node instance with unique, random test data for the +# given test suite (pass the suite's Node `module`). +# +# NOTE: This method does *not* persist the node. To that end, the returned +# instance *won't* have an `_id` property; you should set it if you persist it. +# +# This method is async because it queries Neo4j's version if we haven't already, +# and strips label metadata if we're running against Neo4j <2.1.5, which didn't +# return label metadata to drivers. +# +@createTestNode = (suite, _) => + node = new neo4j.Node + labels: [@TEST_LABEL] + properties: @createTestProperties suite + + @queryDbVersion _ + + if @DB_VERSION_STR < '2.1.5' + node.labels = null + + node + +# +# Creates and returns a new Relationship instance with unique, random test data +# for the given test suite (pass the suite's Node `module`). +# +# NOTE: This method does *not* persist the relationship. To that end, the +# returned instance *won't* have its `_id`, `_fromId` or `_toId` properties set; +# you should set those if you persist this relationship. +# +@createTestRelationship = (suite) => + new neo4j.Relationship + type: @TEST_REL_TYPE + properties: @createTestProperties suite + +# +# Executes a Cypher query to create and persist a test graph with the given +# number of nodes, connected in a chain. +# +# TODO: Support other types of graphs, e.g. networks, fan-outs, etc.? +# +# The nodes are identified by the filename of the given test suite (pass the +# suite's Node `module`). +# +# Returns an array of Node and Relationship instances for the created graph, +# in chain order, e.g. [node, rel, node]. +# +@createTestGraph = (suite, numNodes, _) => + expect(numNodes).to.be.at.least 1 + numRels = numNodes - 1 + + nodes = (@createTestNode suite, _ for i in [0...numNodes]) + rels = (@createTestRelationship suite for i in [0...numRels]) + + nodeProps = $(nodes).pluck 'properties' + relProps = $(rels).pluck 'properties' + + params = {} + for props, i in nodeProps + params["nodeProps#{i}"] = props + for props, i in relProps + params["relProps#{i}"] = props + + query = '' + for node, i in nodes + query += "CREATE (node#{i}:#{@TEST_LABEL} {nodeProps#{i}}) \n" + for rel, i in rels + query += "CREATE (node#{i}) + -[rel#{i}:#{@TEST_REL_TYPE} {relProps#{i}}]-> (node#{i + 1}) \n" + query += 'RETURN ' + query += ("ID(node#{i})" for node, i in nodes).join ', ' + if rels.length + query += ', ' + query += ("ID(rel#{i})" for rel, i in rels).join ', ' + + # NOTE: Using the old Cypher endpoint here. We don't want to rely on this + # driver's Cypher implementation, nor re-implement the (more complex) new + # endpoint here. This does, however, rely on this driver's HTTP support in + # general, but not on its ability to parse nodes and relationships. + # http://neo4j.com/docs/stable/rest-api-cypher.html#rest-api-use-parameters + {data} = @DB.http + method: 'POST' + path: '/db/data/cypher' + body: {query, params} + , _ + + [row] = data + + for node, i in nodes + node._id = row[i] + + for rel, i in rels + rel._id = row[numNodes + i] + rel._fromId = nodes[i]._id + rel._toId = nodes[i + 1]._id + + results = [] + + for node, i in nodes + results.push node + results.push rels[i] if rels[i] + + results + +# +# Executes a Cypher query to delete the test graph created by `@createTestGraph` +# for the given test suite (pass the suite's Node `module`). +# +@deleteTestGraph = (suite, _) => + @DB.http + method: 'POST' + path: '/db/data/cypher' + body: + query: """ + MATCH (node:#{@TEST_LABEL} {suite: {suite}}) + OPTIONAL MATCH (node) \ + -[rel:#{@TEST_REL_TYPE} {suite: {suite}}]-> () + DELETE node, rel + """ + params: @createTestProperties suite + , _ diff --git a/test-new/http._coffee b/test-new/http._coffee index 6bb6162..d2c2df6 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -11,24 +11,9 @@ neo4j = require '../' ## SHARED STATE -{DB, TEST_LABEL, TEST_REL_TYPE} = fixtures +{DB} = fixtures -[DB_VERSION_STR, DB_VERSION_NUM] = [] - -TEST_NODE_A = new neo4j.Node - # _id will get filled in once we persist - labels: [TEST_LABEL] - properties: {suite: module.filename, name: 'a'} - -TEST_NODE_B = new neo4j.Node - # _id will get filled in once we persist - labels: [TEST_LABEL] - properties: {suite: module.filename, name: 'b'} - -TEST_REL = new neo4j.Relationship - # _id, _fromId (node A), _toId (node B) will get filled in once we persist - type: TEST_REL_TYPE - properties: {suite: module.filename, name: 'r'} +[TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] ## HELPERS @@ -184,55 +169,9 @@ describe 'GraphDatabase::http', -> ## Object parsing: - it '(query Neo4j version)', (_) -> - info = DB.http - method: 'GET' - path: '/db/data/' - , _ - - DB_VERSION_STR = info.neo4j_version or '0' - DB_VERSION_NUM = parseFloat DB_VERSION_STR, 10 - - if DB_VERSION_NUM < 2 - throw new Error '*** node-neo4j v2 supports Neo4j v2+ only, - and you’re running Neo4j v1. These tests will fail! ***' - - # Neo4j <2.1.5 didn't return label info, so returned nodes won't have - # the labels we expect. Account for that: - if DB_VERSION_STR < '2.1.5' - TEST_NODE_A.labels = null - TEST_NODE_B.labels = null - - it '(create test objects)', (_) -> - # NOTE: Using the old Cypher endpoint for simplicity here. - # Nicer than using the raw REST API to create these test objects, - # but also nice to neither use this driver's Cypher functionality - # (which is tested in a higher-level test suite), nor re-implement it. - # http://neo4j.com/docs/stable/rest-api-cypher.html#rest-api-use-parameters - {data} = DB.http - method: 'POST' - path: '/db/data/cypher' - body: - query: """ - CREATE (a:#{TEST_LABEL} {propsA}) - CREATE (b:#{TEST_LABEL} {propsB}) - CREATE (a) -[r:#{TEST_REL_TYPE} {propsR}]-> (b) - RETURN ID(a), ID(b), ID(r) - """ - params: - propsA: TEST_NODE_A.properties - propsB: TEST_NODE_B.properties - propsR: TEST_REL.properties - , _ - - [row] = data - [idA, idB, idR] = row - - TEST_NODE_A._id = idA - TEST_NODE_B._id = idB - TEST_REL._id = idR - TEST_REL._fromId = idA - TEST_REL._toId = idB + it '(create test graph)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ it 'should parse nodes by default', (_) -> node = DB.http @@ -286,7 +225,7 @@ describe 'GraphDatabase::http', -> expect(body).to.not.be.an.instanceOf neo4j.Node # NOTE: Neo4j <2.1.5 didn't return `metadata`, so can't rely on it: - if DB_VERSION_STR >= '2.1.5' + if fixtures.DB_VERSION_STR >= '2.1.5' expect(body.metadata).to.be.an 'object' expect(body.metadata.id).to.equal TEST_NODE_A._id expect(body.metadata.labels).to.eql TEST_NODE_A.labels @@ -303,24 +242,12 @@ describe 'GraphDatabase::http', -> expect(body).to.not.be.an.instanceOf neo4j.Relationship # NOTE: Neo4j <2.1.5 didn't return `metadata`, so can't rely on it: - if DB_VERSION_STR >= '2.1.5' + if fixtures.DB_VERSION_STR >= '2.1.5' expect(body.metadata).to.be.an 'object' expect(body.metadata.id).to.equal TEST_REL._id expect(body.type).to.equal TEST_REL.type expect(body.data).to.eql TEST_REL.properties - it '(delete test objects)', (_) -> - DB.http - method: 'POST' - path: '/db/data/cypher' - body: - query: """ - START a = node({idA}), b = node({idB}), r = rel({idR}) - DELETE a, b, r - """ - params: - idA: TEST_NODE_A._id - idB: TEST_NODE_B._id - idR: TEST_REL._id - , _ + it '(delete test graph)', (_) -> + fixtures.deleteTestGraph module, _ From 3cb4f06ed86d8772b9ce4f90388519a9223ae4dc Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 30 Jan 2015 20:16:25 -0500 Subject: [PATCH 021/121] v2 / Transactions: impl. and test thoroughly! --- API_v2.md | 27 +- lib-new/GraphDatabase.coffee | 37 ++- lib-new/Transaction.coffee | 165 +++++++++++ lib-new/index.coffee | 1 + test-new/transactions._coffee | 500 ++++++++++++++++++++++++++++++++++ 5 files changed, 715 insertions(+), 15 deletions(-) create mode 100644 lib-new/Transaction.coffee create mode 100644 test-new/transactions._coffee diff --git a/API_v2.md b/API_v2.md index 0dbc952..5ccdd79 100644 --- a/API_v2.md +++ b/API_v2.md @@ -200,17 +200,24 @@ so it could always be achieved automatically under-the-hood by this driver Please provide feedback if you disagree! ```js -var tx = db.beginTransaction(); // any options needed? +var tx = db.beginTransaction(); ``` -This method returns a `Transaction` object, which mainly just encapsulates the -state of a "transaction ID" returned by Neo4j from the first query. - This method is named "begin" instead of "create" to reflect that it returns immediately, and has not actually persisted anything to the database yet. +This method returns a `Transaction` object, which mainly just encapsulates the +state of a "transaction ID" returned by Neo4j from the first query. +In addition, though, `Transaction` instances expose helpful properties to let +you know a transaction's precise state, e.g. whether it was automatically +rolled back by Neo4j due to a fatal error. +This precise state tracking also lets this driver prevent predictable errors, +which may be signs of code bugs, and provide more helpful error messaging. + ```coffee -class Transaction {_id} +class Transaction {_id, expiresAt, expiresIn, state} +# `expiresAt` is a Date, while `expiresIn` is a millisecond count. +# `state` is one of 'open', 'pending', 'committed, 'rolled back', or 'expired'. ``` ```js @@ -221,18 +228,16 @@ var stream = tx.cypher({query, params, headers, raw, commit}, cbResults); tx.commit(cbDone); tx.rollback(cbDone); +tx.renew(cbDone); ``` The transactional `cypher` method is just like the regular `cypher` method, except that it supports an additional `commit` option, which can be set to -`true` to automatically attempt to commit the transaction after this query. +`true` to automatically attempt to commit the transaction with this query. Otherwise, transactions can be committed and rolled back independently. - -TODO: Any more functionality needed for transactions? -There's a notion of expiry, and the expiry timeout can be reset by making -empty queries; should a notion of auto "renewal" (effectively, a higher -timeout than the default) be built-in for convenience? +They can also be explicitly renewed, as Neo4j expires them if idle too long, +but every query in a transaction implicitly renews the transaction as well. ## Errors diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 2e0afad..3a75082 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -4,8 +4,8 @@ lib = require '../package.json' Node = require './Node' Relationship = require './Relationship' Request = require 'request' +Transaction = require './Transaction' URL = require 'url' -utils = require './utils' module.exports = class GraphDatabase @@ -63,14 +63,38 @@ module.exports = class GraphDatabase cb null, _transform body - cypher: (opts={}, cb) -> + cypher: (opts={}, cb, _tx) -> if typeof opts is 'string' opts = {query: opts} - {query, params, headers, raw} = opts + {query, params, headers, raw, commit, rollback} = opts + + if not _tx and rollback + throw new Error 'Illegal state: rolling back without a transaction!' + + if not _tx?._id and rollback + # No query has been made within transaction yet, so this transaction + # doesn't even exist yet from Neo4j's POV; nothing to do. + cb null, null + return + + if commit and rollback + throw new Error 'Illegal state: both committing and rolling back!' + + if not _tx and commit is false + throw new TypeError 'Can’t refuse to commit without a transaction! + To begin a new transaction without committing, call + `db.beginTransaction()`, and then call `cypher` on that.' + + if not _tx and not query + throw new TypeError 'Query required' method = 'POST' - path = '/db/data/transaction/commit' + method = 'DELETE' if rollback + + path = '/db/data/transaction' + path += "/#{_tx._id}" if _tx?._id + path += "/commit" if commit or not _tx # NOTE: Lowercase 'rest' matters here for parsing. format = if raw then 'row' else 'rest' @@ -92,6 +116,8 @@ module.exports = class GraphDatabase # NOTE: This includes our own errors for non-2xx responses. return cb err + _tx?._updateFromResponse body + {results, errors} = body if errors.length @@ -128,6 +154,9 @@ module.exports = class GraphDatabase cb null, results + beginTransaction: -> + new Transaction @ + ## HELPERS diff --git a/lib-new/Transaction.coffee b/lib-new/Transaction.coffee new file mode 100644 index 0000000..12f3660 --- /dev/null +++ b/lib-new/Transaction.coffee @@ -0,0 +1,165 @@ +errors = require './errors' +utils = require './utils' + +# This value is used to construct a Date instance, and unfortunately, neither +# Infinity nor Number.MAX_VALUE are valid Date inputs. There's also no simple +# max value for Dates either (http://stackoverflow.com/a/11526569/132978), +# so we arbitrarily do one year ahead. Hopefully this doesn't matter. +FAR_FUTURE_MS = Date.now() + 1000 * 60 * 60 * 24 * 365 + + +# http://neo4j.com/docs/stable/rest-api-transactional.html +module.exports = class Transaction + + constructor: (@_db) -> + @_id = null + @_expires = null + @_pending = false + @_committed = false + @_rolledback = false + + Object.defineProperty @::, 'expiresAt', + enumerable: true + get: -> + if @_expires + new Date @_expires + else + # This transaction hasn't been created yet, so far future: + new Date FAR_FUTURE_MS + + Object.defineProperty @::, 'expiresIn', + enumerable: true + get: -> + if @_expires + @expiresAt - (new Date) + else + # This transaction hasn't been created yet, so far future. + # Unlike for the Date instance above, we can be less arbitrary; + # hopefully it's never a problem to return Infinity here. + Infinity + + # + # The state of this transaction. Returns one of the following values: + # + # - open + # - pending (a request is in progress) + # - committed + # - rolled back + # - expired + # + # TODO: Should we make this an enum? Or constants? + # + Object.defineProperty @::, 'state', + get: -> switch + # Order matters here. + # + # E.g. a request could have been made just before the expiry time, + # and we won't know the new expiry time until the server responds. + # + # TODO: The server could also receive it just *after* the expiry + # time, which'll cause it to return an unhelpful `UnknownId` error; + # should we handle that edge case in our `cypher` callback below? + # + when @_pending then 'pending' + when @_committed then 'committed' + when @_rolledback then 'rolled back' + when @expiresIn <= 0 then 'expired' + else 'open' + + cypher: (opts={}, cb) -> + # Check predictable error cases to provide better messaging sooner. + # All of these are `ClientErrors` within the `Transaction` category. + # http://neo4j.com/docs/stable/status-codes.html#_status_codes + errMsg = switch @state + when 'pending' + # This would otherwise throw a `ConcurrentRequest` error. + 'A request within this transaction is currently in progress. + Concurrent requests within a transaction are not allowed.' + when 'expired' + # This would otherwise throw an `UnknownId` error. + 'This transaction has expired. + You can get the expiration time of a transaction through its + `expiresAt` (Date) and `expiresIn` (ms) properties. + To prevent a transaction from expiring, execute any action + or call `renew` before the transaction expires.' + when 'committed' + # This would otherwise throw an `UnknownId` error. + 'This transaction has been committed. + Transactions cannot be reused; begin a new one instead.' + when 'rolled back' + # This would otherwise throw an `UnknownId` error. + 'This transaction has been rolled back. + Transactions get automatically rolled back on any + DatabaseErrors, as well as any errors during a commit. + That includes auto-commit queries (`{commit: true}`). + Transactions cannot be reused; begin a new one instead.' + + if errMsg + # TODO: Should we callback this error instead? (And if so, should we + # `process.nextTick` that call?) + # I *think* that these cases are more likely to be code bugs than + # legitimate runtime errors, so the benefit of throwing sync'ly is + # fail-fast behavior, and more helpful stack traces. + throw new errors.ClientError errMsg + + # The only state we should be in at this point is 'open'. + @_pending = true + @_db.cypher opts, (err, results) => + @_pending = false + + # If this transaction still exists, no state changes for us: + if @_id + return cb err, results + + # Otherwise, this transaction was destroyed -- either committed or + # rolled back -- so update our state accordingly. + # Much easier to derive whether committed than whether rolled back, + # because commits can only happen when explicitly requested. + if opts.commit and not err + @_committed = true + else + @_rolledback = true + + cb err, results + , @ + + commit: (cb) -> + @cypher {commit: true}, cb + + rollback: (cb) -> + @cypher {rollback: true}, cb + + renew: (cb) -> + @cypher {}, cb + + # + # Updates this Transaction instance with data from the given transactional + # response body. + # + _updateFromResponse: (body) -> + if not body + throw new Error 'Unexpected: no transactional response body!' + + {transaction, commit} = body + + if not transaction and not commit + # This transaction has been destroyed (either committed or rolled + # back). Our state will get updated in the `cypher` callback above. + @_id = @_expires = null + return + + if not commit + throw new Error 'Unexpected: transaction object, but no commit URL!' + + if not transaction + throw new Error 'Unexpected: commit URL, but no transaction object!' + + @_expires = new Date transaction.expires + + # Unfortunately, there's no simple `id` property on returned + # transaction objects, so we need to parse it out of a URL. + # The canonical URL is returned via a `Location` header in the + # initial 201 response, but we don't have access to that. + # Fortunately, a `commit` URL is returned on every transaction + # response, and we can parse the ID out of that. + @_id = utils.parseId commit diff --git a/lib-new/index.coffee b/lib-new/index.coffee index 150403b..46e700f 100644 --- a/lib-new/index.coffee +++ b/lib-new/index.coffee @@ -4,5 +4,6 @@ $(exports).extend GraphDatabase: require './GraphDatabase' Node: require './Node' Relationship: require './Relationship' + Transaction: require './Transaction' $(exports).extend require './errors' diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee new file mode 100644 index 0000000..6303fef --- /dev/null +++ b/test-new/transactions._coffee @@ -0,0 +1,500 @@ +# +# Tests for Transaction support, e.g. the ability to make multiple queries, +# across network requests, in a single transaction; commit; rollback; etc. +# + +{expect} = require 'chai' +fixtures = require './fixtures' +neo4j = require '../' + + +## SHARED STATE + +{DB, TEST_LABEL} = fixtures + +[TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] + + + +## HELPERS + +# +# Asserts that the given object is an instance of the proper Neo4j Error +# subclass, representing the given transactional Neo4j error info. +# TODO: Consider consolidating with a similar helper in the `http` test suite. +# +expectError = (err, classification, category, title, message) -> + expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError + expect(err.name).to.equal "neo4j.#{classification}" + expect(err.message).to.equal "[#{category}.#{title}] #{message}" + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j).to.contain + code: "Neo.#{classification}.#{category}.#{title}" + message: message + + +## TESTS + +describe 'Transactions', -> + + it 'should support simple queries', (_) -> + tx = DB.beginTransaction() + + [{foo}] = tx.cypher 'RETURN "bar" AS foo', _ + + expect(foo).to.equal 'bar' + + it 'should convey pending state, and reject concurrent requests', (done) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + fn = -> + tx.cypher 'RETURN "bar" AS foo', cb + expect(tx.state).to.equal 'pending' + + cb = (err, results) -> + try + expect(err).to.not.exist() + expect(tx.state).to.equal 'open' + catch assertionErr + return done assertionErr + done() + + fn() + expect(fn).to.throw neo4j.ClientError, /concurrent requests/i + + it '(create test graph)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ + + it 'should isolate effects', (_) -> + tx = DB.beginTransaction() + + # NOTE: It's important for us to create something new here, rather than + # modify something existing. Otherwise, since we don't explicitly + # rollback our open transaction at the end of this test, Neo4j sits and + # waits for it to expire before returning other queries that touch the + # existing graph -- including our last "delete test graph" step. + # To that end, we test creating a new node here. + + {labels, properties} = fixtures.createTestNode module, _ + + [{node}] = tx.cypher + query: """ + CREATE (node:#{TEST_LABEL} {properties}) + RETURN node + """ + params: {properties} + , _ + + expect(node).to.be.an.instanceOf neo4j.Node + expect(node.properties).to.eql properties + expect(node.labels).to.eql labels + expect(node._id).to.be.a 'number' + + # Outside the transaction, we shouldn't see this newly created node: + results = DB.cypher + query: """ + MATCH (node:#{TEST_LABEL}) + WHERE #{( + # NOTE: Cypher doesn’t support directly comparing nodes and + # property bags, so we have to compare each property. + for prop of properties + "node.#{prop} = {properties}.#{prop}" + ).join ' AND '} + RETURN node + """ + params: {properties} + , _ + + expect(results).to.be.empty() + + it 'should support committing, and reject subsequent requests', (_) -> + tx = DB.beginTransaction() + + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'committing' + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'committing' + + expect(tx.state).to.equal 'open' + tx.commit _ + expect(tx.state).to.equal 'committed' + + expect(-> tx.cypher 'RETURN "bar" AS foo') + .to.throw neo4j.ClientError, /been committed/i + + # Outside of the transaction, we should see this change now: + [{nodeA}] = DB.cypher + query: ''' + START nodeA = node({idA}) + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'committing' + + it 'should support auto-committing', (_) -> + tx = DB.beginTransaction() + + # Rather than test auto-committing on the first query, which doesn't + # actually create a new transaction, auto-commit on the second. + + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'auto-committing' + SET nodeA.i = 1 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'auto-committing' + expect(nodeA.properties.i).to.equal 1 + + expect(tx.state).to.equal 'open' + + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 2 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + commit: true + , _ + + expect(nodeA.properties.test).to.equal 'auto-committing' + expect(nodeA.properties.i).to.equal 2 + + expect(tx.state).to.equal 'committed' + + expect(-> tx.cypher 'RETURN "bar" AS foo') + .to.throw neo4j.ClientError, /been committed/i + + # Outside of the transaction, we should see this change now: + [{nodeA}] = DB.cypher + query: ''' + START nodeA = node({idA}) + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'auto-committing' + expect(nodeA.properties.i).to.equal 2 + + it 'should support rolling back, and reject subsequent requests', (_) -> + tx = DB.beginTransaction() + + [{nodeA}] = tx.cypher + query: ''' + START a = node({idA}) + SET a.test = 'rolling back' + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'rolling back' + + expect(tx.state).to.equal 'open' + tx.rollback _ + expect(tx.state).to.equal 'rolled back' + + expect(-> tx.cypher 'RETURN "bar" AS foo') + .to.throw neo4j.ClientError, /been rolled back/i + + # Back outside this transaction now, the change should *not* be visible: + [{nodeA}] = DB.cypher + query: ''' + START a = node({idA}) + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.not.equal 'rolling back' + + it 'should support rolling back before any commits', (_) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + tx.rollback _ + expect(tx.state).to.equal 'rolled back' + + # NOTE: Skipping this test by default, because it's slow (we have to pause + # one second; see note within) and not really a mission-critical feature. + it.skip 'should support renewing (slow)', (_) -> + tx = DB.beginTransaction() + + [{nodeA}] = tx.cypher + query: ''' + START a = node({idA}) + SET a.test = 'renewing' + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'renewing' + + expect(tx.expiresAt).to.be.an.instanceOf Date + expect(tx.expiresAt).to.be.greaterThan new Date + expect(tx.expiresIn).to.be.a 'number' + expect(tx.expiresIn).to.be.greaterThan 0 + expect(tx.expiresIn).to.equal tx.expiresAt - new Date + + # NOTE: We can't easily test transactions actually expiring (that would + # take too long, and there's no way for the client to shorten the time), + # so we can't test that renewing actually *works* / has an effect. + # We can only test that it *appears* to work / have an effect. + # + # NOTE: Neo4j's expiry appears to have a granularity of one second, + # so to be robust (local requests are frequently faster than that), + # we pause a second first. + + oldExpiresAt = tx.expiresAt + setTimeout _, 1000 # TODO: Provide visual feedback? + + expect(tx.state).to.equal 'open' + tx.renew _ + expect(tx.state).to.equal 'open' + + expect(tx.expiresAt).to.be.an.instanceOf Date + expect(tx.expiresAt).to.be.greaterThan new Date + expect(tx.expiresAt).to.be.greaterThan oldExpiresAt + expect(tx.expiresIn).to.be.a 'number' + expect(tx.expiresIn).to.be.greaterThan 0 + expect(tx.expiresIn).to.equal tx.expiresAt - new Date + + # To prevent Neo4j from hanging at the end waiting for this transaction + # to commit or expire (since it touches the existing graph, and our last + # step is to delete the existing graph), roll this transaction back. + tx.rollback _ + expect(tx.state).to.equal 'rolled back' + + # We also ensure that renewing didn't cause the transaction to commit. + [{nodeA}] = DB.cypher + query: ''' + START a = node({idA}) + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.not.equal 'renewing' + + it 'should properly handle non-fatal errors', (_) -> + tx = DB.beginTransaction() + + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'non-fatal errors' + SET nodeA.i = 1 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'non-fatal errors' + expect(nodeA.properties.i).to.equal 1 + + # Now trigger a client error, which should *not* rollback (and thus + # destroy) the transaction. + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 2 + RETURN {foo} + ''' + params: + idA: TEST_NODE_A._id + , (err, results) => + try + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'open' + + # Because of that, the first query's effects should still be visible + # within the transaction: + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 3 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'non-fatal errors' + expect(nodeA.properties.i).to.equal 3 + + # NOTE: But the transaction won't commit successfully apparently, both + # manually or automatically. So we manually rollback instead. + # TODO: Is this a bug in Neo4j? Or my understanding? + expect(tx.state).to.equal 'open' + tx.rollback _ + expect(tx.state).to.equal 'rolled back' + + # TODO: Similar to the note above this, is this right? Or is this either a + # bug in Neo4j or my understanding? Should client errors never be fatal? + it 'should properly handle fatal client errors during commit', (_) -> + tx = DB.beginTransaction() + + # Important: don't auto-commit in the first query, because that doesn't + # let us test that a transaction gets *returned* and *then* rolled back. + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'fatal client errors' + SET nodeA.i = 1 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'fatal client errors' + expect(nodeA.properties.i).to.equal 1 + + # Now trigger a client error in an auto-commit query, which *should* + # (apparently; see comment preceding test) destroy the transaction. + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 2 + RETURN {foo} + ''' + params: + idA: TEST_NODE_A._id + commit: true + , (err, results) => + try + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'rolled back' + + expect(-> tx.cypher 'RETURN "bar" AS foo') + .to.throw neo4j.ClientError, /been rolled back/i + + # Back outside this transaction now, the change should *not* be visible: + [{nodeA}] = DB.cypher + query: ''' + START a = node({idA}) + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.not.equal 'fatal client errors' + + it 'should properly handle fatal database errors', (_) -> + tx = DB.beginTransaction() + + # Important: don't auto-commit in the first query, because that doesn't + # let us test that a transaction gets *returned* and *then* rolled back. + [{nodeA}] = tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'fatal database errors' + SET nodeA.i = 1 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.equal 'fatal database errors' + expect(nodeA.properties.i).to.equal 1 + + # The only way I know how to trigger a database error is to trigger a + # client error, and then *separately* attempt to commit the transaction. + # TODO: Is there any better way? + try + tx.cypher + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 2 + RETURN {foo} + ''' + params: + idA: TEST_NODE_A._id + , _ + catch err + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.commit (err) => + try + expect(err).to.exist() + expectError err, 'DatabaseError', 'Transaction', + 'CouldNotCommit', 'javax.transaction.RollbackException: + Failed to commit, transaction rolled back' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'rolled back' + + expect(-> tx.cypher 'RETURN "bar" AS foo') + .to.throw neo4j.ClientError, /been rolled back/i + + # The change should thus *not* be visible back outside the transaction: + [{nodeA}] = DB.cypher + query: ''' + START a = node({idA}) + RETURN a AS nodeA + ''' + params: + idA: TEST_NODE_A._id + , _ + + expect(nodeA.properties.test).to.not.equal 'fatal database errors' + + # TODO: Is there any way to trigger and test transient errors? + + it 'should support streaming (TODO)' + + it '(delete test graph)', (_) -> + fixtures.deleteTestGraph module, _ From edf7ab45a35e65e10915ae42452a3f98946f7667 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Feb 2015 19:08:55 -0500 Subject: [PATCH 022/121] v2 / Batching: spec out, impl., and add tests! --- API_v2.md | 53 ++++++------ lib-new/GraphDatabase.coffee | 148 ++++++++++++++++++++++++---------- test-new/cypher._coffee | 119 ++++++++++++++++++++++++++- test-new/transactions._coffee | 118 +++++++++++++++++++++++++++ 4 files changed, 372 insertions(+), 66 deletions(-) diff --git a/API_v2.md b/API_v2.md index 5ccdd79..66c0797 100644 --- a/API_v2.md +++ b/API_v2.md @@ -172,33 +172,39 @@ Or should we just link to cypher-stream? TODO: Should we allow access to other underlying data formats, e.g. "graph"? +## Batching + +**Let me make multiple, separate Cypher queries in one network request.** + +```js +function cbMany(err, arraysOfResults) {} + +var streams = db.cypher({queries, headers}, cbMany); + +// Alternate simple version -- no custom headers: +var streams = db.cypher(queries, cbMany); +``` + +In both cases, `queries` is an array of queries, where each query can be a +`{query, params, raw}` object or a simple string. + +**Important:** batch queries are executed transactionally — +either they all succeed, or they all fail. + +If a callback is given, it'll be called with an array containing an array of +results for each query (in matching order), or an error if there is any. +Alternately, the results can be streamed back by omitting the callback. +In that case, an array will be returned, containing a stream for each query +(in matching order). + +TODO: Is it valuable to return a stream of results *across all queries*? + + ## Transactions **Let me make multiple queries, across multiple network requests, all within a single transaction.** -This is the trickiest part of the API to design. -I've tried my best to design this using the use cases we have at FiftyThree, -but it's very hard to know whether this is designed well for a broader set of -use cases without having more experience or feedback. - -Example use case: complex delete. -I want to delete an image, which has some image-specific business logic, -but in addition, I need to delete any likes and comments on the image. -Each of those has its own specific business logic (which may also be -recursive), so our code can't capture everything in a single query. -Thus, we need to make one query to delete the comments and likes (which may -actually be multiple queries, as well), then a second one to delete the image. -We want to do all of that transactionally, so that if any one query fails, -we abort/rollback and either retry or report failure to the user. - -Given a use case like that, this API is optimized for **one query per network -request**, *not* multiple queries per network request ("batching"). -I *think* batching is always an optimization (never a true *requirement*), -so it could always be achieved automatically under-the-hood by this driver -(e.g. by waiting until the next event loop tick to send the actual queries). -Please provide feedback if you disagree! - ```js var tx = db.beginTransaction(); ``` @@ -231,7 +237,8 @@ tx.rollback(cbDone); tx.renew(cbDone); ``` -The transactional `cypher` method is just like the regular `cypher` method, +The transactional `cypher` method is just like the regular `cypher` method +(e.g. it supports simple strings, as well as batching), except that it supports an additional `commit` option, which can be set to `true` to automatically attempt to commit the transaction with this query. diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 3a75082..59ca5e1 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,4 +1,5 @@ $ = require 'underscore' +assert = require 'assert' {Error} = require './errors' lib = require '../package.json' Node = require './Node' @@ -67,7 +68,10 @@ module.exports = class GraphDatabase if typeof opts is 'string' opts = {query: opts} - {query, params, headers, raw, commit, rollback} = opts + if opts instanceof Array + opts = {queries: opts} + + {queries, query, params, headers, raw, commit, rollback} = opts if not _tx and rollback throw new Error 'Illegal state: rolling back without a transaction!' @@ -86,8 +90,20 @@ module.exports = class GraphDatabase To begin a new transaction without committing, call `db.beginTransaction()`, and then call `cypher` on that.' - if not _tx and not query - throw new TypeError 'Query required' + if not _tx and not query and not queries + throw new TypeError 'Query or queries required' + + if query and queries + throw new TypeError 'Can’t supply both a single query + and a batch of queries! Do you have a bug in your code?' + + if queries and params + throw new TypeError 'When batching multiple queries, + params must be supplied with each query, not globally.' + + if queries and raw + throw new TypeError 'When batching multiple queries, + `raw` must be specified with each query, not globally.' method = 'POST' method = 'DELETE' if rollback @@ -96,17 +112,40 @@ module.exports = class GraphDatabase path += "/#{_tx._id}" if _tx?._id path += "/commit" if commit or not _tx - # NOTE: Lowercase 'rest' matters here for parsing. - format = if raw then 'row' else 'rest' - statements = [] - body = {statements} - - # TODO: Support batching multiple queries in this request? + # Normalize input query or queries to an array of queries always, + # but remember whether a single query was given (not a batch). + # Also handle the case where no queries were given; this is either a + # void action (e.g. rollback), or legitimately an empty batch. if query - statements.push - statement: query - parameters: params or {} - resultDataContents: [format] + queries = [{query, params, raw}] + single = true + else + single = not queries # void action, *not* empty [] given + queries or= [] + + # Generate the request body by transforming each query (which is + # potentially a simple string) into Neo4j's `statement` format. + # We need to remember what result format we requested for each query. + formats = [] + body = + statements: + for query in queries + if typeof query is 'string' + query = {query} + + if query.headers + throw new TypeError 'When batching multiple queries, + custom request headers cannot be supplied per query; + they must be supplied globally.' + + {query, params, raw} = query + + # NOTE: Lowercase 'rest' matters here for parsing. + formats.push format = if raw then 'row' else 'rest' + + statement: query + parameters: params or {} + resultDataContents: [format] # TODO: Support streaming! @http {method, path, headers, body}, (err, body) => @@ -120,39 +159,66 @@ module.exports = class GraphDatabase {results, errors} = body + # Parse any results first, before errors, in case this is a batch + # request, where we want to return results alongside errors. + # The top-level `results` is an array of results corresponding to + # the `statements` (queries) inputted. + # We want to transform each query's results from Neo4j's complex + # format to a simple array of dictionaries. + results = + for result, i in results + {columns, data} = result + format = formats[i] + + # The `data` for each query is an array of rows, but each of + # its elements is actually a dictionary of results keyed by + # response format. We only request one format per query. + # The value of each format is an array of rows, where each + # row is an array of column values. We transform those rows + # into dictionaries keyed by column names. Phew! + $(data).pluck(format).map (row) -> + result = {} + for column, j in columns + result[column] = row[j] + result + + # What exactly we return depends on how we were called: + # + # - Batch: if an array of queries were given, we always return an + # array of each query's results. + # + # - Single: if a single query was given, we always return just that + # query's results. + # + # - Void: if neither was given, we explicitly return null. + # This is for transaction actions, e.g. commit, rollback, renew. + # + # We're already set up for the batch case by default, so we only + # need to account for the other cases. + # + if single + # This means a batch of queries was *not* given, but we still + # normalized to an array of queries... + if queries.length + # This means a single query was given: + assert.equal queries.length, 1, + 'There should be *exactly* one query given.' + assert results.length <= 1, + 'There should be *at most* one set of results.' + results = results[0] + else + # This means no query was given: + assert.equal results.length, 0, + 'There should be *no* results.' + results = null + if errors.length # TODO: Is it possible to get back more than one error? # If so, is it fine for us to just use the first one? [error] = errors - return cb Error._fromTransaction error - - # If there are no results, it means no statements were sent - # (e.g. to commit, rollback, or renew a transaction in isolation), - # so nothing to return, i.e. a void call in that case. - # Important: we explicitly don't return an empty array, because that - # implies we *did* send a query, that just didn't match anything. - if not results.length - return cb null, null + err = Error._fromTransaction error - # The top-level `results` is an array of results corresponding to - # the `statements` (queries) inputted. - # We send only one statement/query, so we have only one result. - [result] = results - {columns, data} = result - - # The `data` is an array of result rows, but each of its elements is - # actually a dictionary of results keyed by *response format*. - # We only request one format, `rest` by default, `row` if `raw`. - # In both cases, the value is an array of rows, where each row is an - # array of column values. - # We transform those rows into dictionaries keyed by column names. - results = $(data).pluck(format).map (row) -> - result = {} - for column, i in columns - result[column] = row[i] - result - - cb null, results + cb err, results beginTransaction: -> new Transaction @ diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 9e976d6..723627f 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -110,11 +110,37 @@ describe 'GraphDatabase::cypher', -> DB.cypher 'RETURN {foo}', (err, results) -> try expect(err).to.exist() - expect(results).to.not.exist() - expectError err, 'ClientError', 'Statement', 'ParameterMissing', 'Expected a parameter named foo' + # Whether `results` are returned or not depends on the error; + # Neo4j will return an array if the query could be executed, + # and then it'll return whatever results it could manage to get + # before the error. In this case, the query began execution, + # so we expect an array, but no actual results. + expect(results).to.be.an 'array' + expect(results).to.be.empty() + + catch assertionErr + return done assertionErr + + done() + + it 'should properly return null result on syntax errors', (done) -> + DB.cypher '(syntax error)', (err, results) -> + try + expect(err).to.exist() + + # Simplified error checking, since the message is complex: + expect(err).to.be.an.instanceOf neo4j.ClientError + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.code).to.equal \ + 'Neo.ClientError.Statement.InvalidSyntax' + + # Unlike the previous test case, since Neo4j could not be + # executed, no results should have been returned at all: + expect(results).to.not.exist() + catch assertionErr return done assertionErr @@ -188,6 +214,95 @@ describe 'GraphDatabase::cypher', -> r: TEST_REL.properties ] + it 'should support simple batching', (_) -> + results = DB.cypher [ + query: ''' + START a = node({idA}) + RETURN a + ''' + params: + idA: TEST_NODE_A._id + , + query: ''' + START b = node({idB}) + RETURN b + ''' + params: + idB: TEST_NODE_B._id + , + query: ''' + START r = rel({idR}) + RETURN r + ''' + params: + idR: TEST_REL._id + ], _ + + expect(results).to.be.an 'array' + expect(results).to.have.length 3 + + [resultsA, resultsB, resultsR] = results + + expect(resultsA).to.eql [ + a: TEST_NODE_A + ] + + expect(resultsB).to.eql [ + b: TEST_NODE_B + ] + + expect(resultsR).to.eql [ + r: TEST_REL + ] + + it 'should handle complex batching with errors', (done) -> + DB.cypher + queries: [ + query: ''' + START a = node({idA}) + RETURN a + ''' + params: + idA: TEST_NODE_A._id + raw: true + , + 'RETURN {foo}' + , + query: ''' + START r = rel({idR}) + RETURN r + ''' + params: + idR: TEST_REL._id + ] + , (err, results) -> + try + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', 'ParameterMissing', + 'Expected a parameter named foo' + + # NOTE: With batching, we *do* return any results that we + # received before the error, in case of an open transaction. + # This means that we'll always return an array here, and it'll + # have just as many elements as queries that returned an array + # before the error. In this case, a ParameterMissing error in + # the second query means the second array *was* returned (since + # Neo4j could begin executing the query; see note in the first + # error handling test case in this suite), so two results. + expect(results).to.be.an 'array' + expect(results).to.have.length 2 + + [resultsA, resultsB] = results + expect(resultsA).to.eql [ + a: TEST_NODE_A.properties + ] + expect(resultsB).to.be.empty() + + catch assertionErr + return done assertionErr + + done() + it 'should support streaming (TODO)' it '(delete test graph)', (_) -> diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 6303fef..b91ee55 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -494,6 +494,124 @@ describe 'Transactions', -> # TODO: Is there any way to trigger and test transient errors? + it 'should properly handle errors with batching', (_) -> + tx = DB.beginTransaction() + + results = tx.cypher [ + query: ''' + START nodeA = node({idA}) + SET nodeA.test = 'errors with batching' + ''' + params: + idA: TEST_NODE_A._id + , + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 1 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + ], _ + + expect(results).to.be.an 'array' + expect(results).to.have.length 2 + + for result in results + expect(result).to.be.an 'array' + + expect(results[0]).to.be.empty() + expect(results[1]).to.have.length 1 + + [{nodeA}] = results[1] + + expect(nodeA.properties.test).to.equal 'errors with batching' + expect(nodeA.properties.i).to.equal 1 + + expect(tx.state).to.equal 'open' + + # Now trigger a client error within another batch; this should *not* + # rollback (and thus destroy) the transaction. + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher + queries: [ + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 2 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + raw: true + , + '(syntax error)' + , + query: ''' + START nodeA = node({idA}) + SET nodeA.i = 3 + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + ] + , (err, results) => + try + expect(err).to.exist() + + # Simplified error checking, since the message is complex: + expect(err).to.be.an.instanceOf neo4j.ClientError + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.code).to.equal \ + 'Neo.ClientError.Statement.InvalidSyntax' + + expect(results).to.be.an 'array' + expect(results).to.have.length 1 + + [result] = results + + expect(result).to.be.an 'array' + expect(result).to.have.length 1 + + [{nodeA}] = result + + # We requested `raw: true`, so `nodeA` is just properties: + expect(nodeA.test).to.equal 'errors with batching' + expect(nodeA.i).to.equal 2 + + catch assertionErr + return cont assertionErr + + cont() + + expect(tx.state).to.equal 'open' + + # Because of that, the effects of the first query in the batch (before + # the error) should still be visible within the transaction: + results = tx.cypher [ + query: ''' + START nodeA = node({idA}) + RETURN nodeA + ''' + params: + idA: TEST_NODE_A._id + ], _ + + expect(results).to.be.an 'array' + expect(results).to.have.length 1 + + [{nodeA}] = results[0] + + expect(nodeA.properties.test).to.equal 'errors with batching' + expect(nodeA.properties.i).to.equal 2 + + # NOTE: But the transaction won't commit successfully apparently, both + # manually or automatically. So we manually rollback instead. + # TODO: Is this a bug in Neo4j? Or my understanding? + expect(tx.state).to.equal 'open' + tx.rollback _ + expect(tx.state).to.equal 'rolled back' + it 'should support streaming (TODO)' it '(delete test graph)', (_) -> From 6aa2ff6c7680e8c15e2e8ae567e2668cde1f7af9 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Feb 2015 20:45:06 -0500 Subject: [PATCH 023/121] v2 / Cypher: more efficient impl. for raw queries. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No need to traverse the response body looking for REST format nodes & relationships when there won’t be any. --- lib-new/GraphDatabase.coffee | 28 ++++++++++++++++++------- lib-new/Transaction.coffee | 40 ++++++++++++++++++++---------------- 2 files changed, 43 insertions(+), 25 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 59ca5e1..061d616 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -57,12 +57,10 @@ module.exports = class GraphDatabase # TODO: Do we want to return our own Response object? return cb null, resp - {body, headers, statusCode} = resp - if err = Error._fromResponse resp return cb err - cb null, _transform body + cb null, _transform resp.body cypher: (opts={}, cb, _tx) -> if typeof opts is 'string' @@ -148,16 +146,25 @@ module.exports = class GraphDatabase resultDataContents: [format] # TODO: Support streaming! - @http {method, path, headers, body}, (err, body) => + # + # NOTE: Specifying `raw: true` to `http` to save on parsing work + # (see `_transform` helper at the bottom of this file) if `raw: true` + # was specified to *us* (so no parsing of nodes & rels is needed). + # Easy enough for us to parse ourselves, which we do, when needed. + # + @http {method, path, headers, body, raw: true}, (err, resp) => if err # TODO: Do we want to wrap or modify native errors? # NOTE: This includes our own errors for non-2xx responses. return cb err - _tx?._updateFromResponse body + if err = Error._fromResponse resp + return cb err + + _tx?._updateFromResponse resp - {results, errors} = body + {results, errors} = resp.body # Parse any results first, before errors, in case this is a batch # request, where we want to return results alongside errors. @@ -175,11 +182,18 @@ module.exports = class GraphDatabase # response format. We only request one format per query. # The value of each format is an array of rows, where each # row is an array of column values. We transform those rows - # into dictionaries keyed by column names. Phew! + # into dictionaries keyed by column names. Finally, we also + # parse nodes & relationships into object instances if this + # query didn't request a raw format. Phew! $(data).pluck(format).map (row) -> result = {} + for column, j in columns result[column] = row[j] + + if format is 'rest' + result = _transform result + result # What exactly we return depends on how we were called: diff --git a/lib-new/Transaction.coffee b/lib-new/Transaction.coffee index 12f3660..18092d5 100644 --- a/lib-new/Transaction.coffee +++ b/lib-new/Transaction.coffee @@ -134,32 +134,36 @@ module.exports = class Transaction # # Updates this Transaction instance with data from the given transactional - # response body. + # endpoint response. # - _updateFromResponse: (body) -> - if not body - throw new Error 'Unexpected: no transactional response body!' + _updateFromResponse: (resp) -> + if not resp + throw new Error 'Unexpected: no transactional response!' - {transaction, commit} = body + {body, headers, statusCode} = resp + {transaction} = body - if not transaction and not commit + if not transaction # This transaction has been destroyed (either committed or rolled # back). Our state will get updated in the `cypher` callback above. @_id = @_expires = null return - if not commit - throw new Error 'Unexpected: transaction object, but no commit URL!' + # Otherwise, this transaction exists. + # The returned object always includes an updated expiry time... + @_expires = new Date transaction.expires - if not transaction - throw new Error 'Unexpected: commit URL, but no transaction object!' + # ...but only includes the URL (from which we can parse its ID) + # the first time, via a Location header for a 201 Created response. + # We can short-circuit if we already have our ID. + return if @_id - @_expires = new Date transaction.expires + if statusCode isnt 201 + throw new Error 'Unexpected: transaction returned by Neo4j, + but it was never 201 Created, so we have no ID!' + + if not transactionURL = headers['location'] + throw new Error 'Unexpected: transaction response is 201 Created, + but with no Location header!' - # Unfortunately, there's no simple `id` property on returned - # transaction objects, so we need to parse it out of a URL. - # The canonical URL is returned via a `Location` header in the - # initial 201 response, but we don't have access to that. - # Fortunately, a `commit` URL is returned on every transaction - # response, and we can parse the ID out of that. - @_id = utils.parseId commit + @_id = utils.parseId transactionURL From f7d29db4586ff1c53b4d7cb1cf1894032ba0b4e6 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Feb 2015 20:48:32 -0500 Subject: [PATCH 024/121] v2 / Cypher: change API from `raw` to `lean`! Clearer and more precise name. --- API_v2.md | 10 +++++----- lib-new/GraphDatabase.coffee | 17 ++++++++--------- test-new/cypher._coffee | 6 +++--- test-new/transactions._coffee | 4 ++-- 4 files changed, 18 insertions(+), 19 deletions(-) diff --git a/API_v2.md b/API_v2.md index 66c0797..a7cd3c7 100644 --- a/API_v2.md +++ b/API_v2.md @@ -146,9 +146,9 @@ from database responses. ```js function cb(err, results) {}; -var stream = db.cypher({query, params, headers, raw}, cb); +var stream = db.cypher({query, params, headers, lean}, cb); -// Alternate simple version -- no params, no headers, not raw: +// Alternate simple version -- no params, no headers, not lean: var stream = db.cypher(query, cb); ``` @@ -163,7 +163,7 @@ row value for that column. In addition, by default, nodes and relationships will be transformed to `Node` and `Relationship` objects. If you don't need the full knowledge of node and relationship metadata -(labels, types, native IDs), you can bypass that by specifying `raw: true`, +(labels, types, native IDs), you can bypass that by specifying `lean: true`, which will return just property data, for a potential performance gain. TODO: Should we formalize the streaming case into a documented Stream class? @@ -186,7 +186,7 @@ var streams = db.cypher(queries, cbMany); ``` In both cases, `queries` is an array of queries, where each query can be a -`{query, params, raw}` object or a simple string. +`{query, params, lean}` object or a simple string. **Important:** batch queries are executed transactionally — either they all succeed, or they all fail. @@ -230,7 +230,7 @@ class Transaction {_id, expiresAt, expiresIn, state} function cbResults(err, results) {}; function cbDone(err) {}; -var stream = tx.cypher({query, params, headers, raw, commit}, cbResults); +var stream = tx.cypher({query, params, headers, lean, commit}, cbResults); tx.commit(cbDone); tx.rollback(cbDone); diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 061d616..8ab7ee3 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -69,7 +69,7 @@ module.exports = class GraphDatabase if opts instanceof Array opts = {queries: opts} - {queries, query, params, headers, raw, commit, rollback} = opts + {queries, query, params, headers, lean, commit, rollback} = opts if not _tx and rollback throw new Error 'Illegal state: rolling back without a transaction!' @@ -99,9 +99,9 @@ module.exports = class GraphDatabase throw new TypeError 'When batching multiple queries, params must be supplied with each query, not globally.' - if queries and raw + if queries and lean throw new TypeError 'When batching multiple queries, - `raw` must be specified with each query, not globally.' + `lean` must be specified with each query, not globally.' method = 'POST' method = 'DELETE' if rollback @@ -115,7 +115,7 @@ module.exports = class GraphDatabase # Also handle the case where no queries were given; this is either a # void action (e.g. rollback), or legitimately an empty batch. if query - queries = [{query, params, raw}] + queries = [{query, params, lean}] single = true else single = not queries # void action, *not* empty [] given @@ -136,10 +136,10 @@ module.exports = class GraphDatabase custom request headers cannot be supplied per query; they must be supplied globally.' - {query, params, raw} = query + {query, params, lean} = query # NOTE: Lowercase 'rest' matters here for parsing. - formats.push format = if raw then 'row' else 'rest' + formats.push format = if lean then 'row' else 'rest' statement: query parameters: params or {} @@ -147,9 +147,8 @@ module.exports = class GraphDatabase # TODO: Support streaming! # - # NOTE: Specifying `raw: true` to `http` to save on parsing work - # (see `_transform` helper at the bottom of this file) if `raw: true` - # was specified to *us* (so no parsing of nodes & rels is needed). + # NOTE: Specifying `raw: true` to save on parsing work (see `_transform` + # helper at the bottom of this file) if any queries are `lean: true`. # Easy enough for us to parse ourselves, which we do, when needed. # @http {method, path, headers, body, raw: true}, (err, resp) => diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 723627f..0a34c6c 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -196,7 +196,7 @@ describe 'GraphDatabase::cypher', -> expect(results[1].outer[0].inner).to.be.an.instanceOf neo4j.Node expect(results[2].outer[0].inner).to.be.an.instanceOf neo4j.Relationship - it 'should not parse nodes & relationships if raw', (_) -> + it 'should not parse nodes & relationships if lean', (_) -> results = DB.cypher query: """ START a = node({idA}) @@ -205,7 +205,7 @@ describe 'GraphDatabase::cypher', -> """ params: idA: TEST_NODE_A._id - raw: true + lean: true , _ expect(results).to.eql [ @@ -264,7 +264,7 @@ describe 'GraphDatabase::cypher', -> ''' params: idA: TEST_NODE_A._id - raw: true + lean: true , 'RETURN {foo}' , diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index b91ee55..c219e08 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -543,7 +543,7 @@ describe 'Transactions', -> ''' params: idA: TEST_NODE_A._id - raw: true + lean: true , '(syntax error)' , @@ -575,7 +575,7 @@ describe 'Transactions', -> [{nodeA}] = result - # We requested `raw: true`, so `nodeA` is just properties: + # We requested `lean: true`, so `nodeA` is just properties: expect(nodeA.test).to.equal 'errors with batching' expect(nodeA.i).to.equal 2 From 9ebfe98aef611dbd60a386df5df2fb046289be2d Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Feb 2015 20:52:16 -0500 Subject: [PATCH 025/121] v2 / Tests: reduce Mocha time thresholds. We no longer have such slow tests. Woohoo! --- test/mocha.opts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/mocha.opts b/test/mocha.opts index 80e0c18..165c6e0 100644 --- a/test/mocha.opts +++ b/test/mocha.opts @@ -1,4 +1,4 @@ --compilers coffee:coffee-script/register,_coffee:streamline/register --reporter spec ---timeout 10000 ---slow 1000 +--timeout 5000 +--slow 500 From 5c6ffc0a532627d42fda4a0db703126291f057ad Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Feb 2015 22:28:34 -0500 Subject: [PATCH 026/121] v2 / HTTP: impl. and test streaming! --- API_v2.md | 17 +++++----- lib-new/GraphDatabase.coffee | 13 ++++++-- test-new/fixtures/fake.json | 7 +++++ test-new/http._coffee | 61 +++++++++++++++++++++++++++++++++--- 4 files changed, 82 insertions(+), 16 deletions(-) create mode 100644 test-new/fixtures/fake.json diff --git a/API_v2.md b/API_v2.md index a7cd3c7..9000928 100644 --- a/API_v2.md +++ b/API_v2.md @@ -74,8 +74,13 @@ function cb(err, body) {}; var req = db.http({method, path, headers, body, raw}, cb); ``` -This method will immediately return a duplex HTTP stream, to and from which -both request and response body data can be piped or streamed. +This method will immediately return a native HTTP [`ClientRequest`][], +to which request data can be streamed, and which emits a `'response'` event +yielding a native HTTP [`IncomingMessage`][], from which (raw chunks of) +response data can be streamed. + +[`ClientRequest`]: http://nodejs.org/api/http.html#http_class_http_clientrequest +[`IncomingMessage`]: http://nodejs.org/api/http.html#http_http_incomingmessage In addition, if a callback is given, it will be called with the final result. By default, this result will be the HTTP response body (parsed as JSON), @@ -90,14 +95,6 @@ The `body` will still be parsed as JSON, but nodes, relationships, and errors will *not* be transformed to node-neo4j objects in this case. In addition, `4xx` and `5xx` status code will *not* yield an error. -Importantly, we don't want to leak the implementation details of which HTTP -library we use. Both [request](https://github.com/request/request) and -[SuperAgent](http://visionmedia.github.io/superagent/#piping-data) are great; -it'd be nice to experiment with both (e.g. SuperAgent supports the browser). -Does this mean we should do anything special when returning HTTP responses? -E.g. should we document our own minimal HTTP `Response` interface that's the -common subset of both libraries? - ## Objects diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 8ab7ee3..c4b1938 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -41,14 +41,16 @@ module.exports = class GraphDatabase method or= 'GET' headers or= {} - # TODO: Do we need to do anything special to support streaming response? req = Request method: method url: URL.resolve @url, path headers: $(headers).defaults @headers json: body ? true - , (err, resp) => + # Important: only pass a callback to Request if a callback was passed + # to us. This prevents Request from doing unnecessary JSON parse work + # if the caller prefers to stream the response instead of buffer it. + , cb and (err, resp) => if err # TODO: Do we want to wrap or modify native errors? return cb err @@ -62,6 +64,13 @@ module.exports = class GraphDatabase cb null, _transform resp.body + # Instead of leaking our (third-party) Request instance, make sure to + # explicitly return only its internal native ClientRequest instance. + # https://github.com/request/request/blob/v2.53.1/request.js#L904 + # This is only populated when the request is `start`ed, so `start` it! + req.start() + req.req + cypher: (opts={}, cb, _tx) -> if typeof opts is 'string' opts = {query: opts} diff --git a/test-new/fixtures/fake.json b/test-new/fixtures/fake.json new file mode 100644 index 0000000..8014f43 --- /dev/null +++ b/test-new/fixtures/fake.json @@ -0,0 +1,7 @@ +{ + "boolean": true, + "number": 1234, + "string": "foo bar baz", + "array": ["foo", "bar", "baz"], + "object": {"foo": {"bar": "baz"}} +} diff --git a/test-new/http._coffee b/test-new/http._coffee index d2c2df6..0e180d3 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -5,6 +5,7 @@ {expect} = require 'chai' fixtures = require './fixtures' +fs = require 'fs' http = require 'http' neo4j = require '../' @@ -161,10 +162,62 @@ describe 'GraphDatabase::http', -> done() - it 'should support streaming (TODO)' - # Test that it immediately returns a duplex HTTP stream. - # Test writing request data to this stream. - # Test reading response data from this stream. + it 'should support streaming', (done) -> + opts = + method: 'PUT' + path: '/db/data/node/-1/properties' + headers: + # Node recommends setting this property when streaming: + # http://nodejs.org/api/http.html#http_request_write_chunk_encoding_callback + 'Transfer-Encoding': 'chunked' + 'X-Foo': 'Bar' # This one's just for testing/verifying + + req = DB.http opts + + expect(req).to.be.an.instanceOf http.ClientRequest + expect(req.method).to.equal opts.method + expect(req.path).to.equal opts.path + + # Special-case for headers since they're stored differently: + for name, val of opts.headers + expect(req.getHeader name).to.equal val + + # Native errors are emitted on this request, so fail-fast if any: + req.on 'error', done + + # Now stream some fake JSON to the request: + fs.createReadStream "#{__dirname}/fixtures/fake.json" + .pipe req + + # Verify that the request fully waits for our stream to finish + # before returning a response: + finished = false + req.on 'finish', -> finished = true + + # When the response is received, stream down its JSON too: + req.on 'response', (resp) -> + expect(finished).to.be.true() + expectResponse resp, 404 + + resp.setEncoding 'utf8' + body = '' + + resp.on 'data', (str) -> body += str + resp.on 'error', done + resp.on 'close', -> done new Error 'Response closed!' + resp.on 'end', -> + try + body = JSON.parse body + + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'NodeNotFoundException' + expect(body.stacktrace).to.be.an 'array' + + catch err + return done err + + done() ## Object parsing: From bd1f80fc14e7c5ebbda8155e34a477ced3b5dc2e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Feb 2015 22:31:29 -0500 Subject: [PATCH 027/121] v2 / Misc: update version, since breaking change. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 120fbd0..285a860 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-alpha1", + "version": "2.0.0-alpha2", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From 6b250210602d40d15e113f94d70fca8720ca651a Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 9 Feb 2015 21:37:54 -0500 Subject: [PATCH 028/121] v2 / HTTP: add support for custom agent, proxy, gzip. --- API_v2.md | 3 ++- lib-new/GraphDatabase.coffee | 7 ++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/API_v2.md b/API_v2.md index 9000928..dc928ac 100644 --- a/API_v2.md +++ b/API_v2.md @@ -45,7 +45,8 @@ var neo4j = require('neo4j'); var db = new neo4j.GraphDatabase({ url: 'http://localhost:7474', headers: {}, // optional defaults, e.g. User-Agent - proxy: '', // optional + proxy: '', // optional URL + agent: null, // optional http.Agent instance, for custom socket pooling }); ``` diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index c4b1938..27db83d 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -20,7 +20,7 @@ module.exports = class GraphDatabase if typeof opts is 'string' opts = {url: opts} - {@url, @headers, @proxy} = opts + {@url, @headers, @proxy, @agent} = opts if not @url throw new TypeError 'URL to Neo4j required' @@ -41,11 +41,16 @@ module.exports = class GraphDatabase method or= 'GET' headers or= {} + # TODO: Would be good to test custom proxy and agent, but difficult. + # Same with Neo4j returning gzipped responses (e.g. through an LB). req = Request method: method url: URL.resolve @url, path + proxy: @proxy headers: $(headers).defaults @headers + agent: @agent json: body ? true + gzip: true # This is only for responses: decode if gzipped. # Important: only pass a callback to Request if a callback was passed # to us. This prevents Request from doing unnecessary JSON parse work From da231fff586a3d04ab66f6caf593e0569c971aa0 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Feb 2015 00:01:50 -0500 Subject: [PATCH 029/121] v2 / Schema: impl. and test labels and misc. --- lib-new/GraphDatabase.coffee | 37 ++++++++++++++++++++++++++ test-new/schema._coffee | 51 ++++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+) create mode 100644 test-new/schema._coffee diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 27db83d..8d1427a 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -251,6 +251,43 @@ module.exports = class GraphDatabase new Transaction @ + ## SCHEMA + + getLabels: (cb) -> + # This endpoint returns the array of labels directly: + # http://neo4j.com/docs/stable/rest-api-node-labels.html#rest-api-list-all-labels + # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + # TODO: Would it be better for us to handle other non-200 responses too? + @http + method: 'GET' + path: '/db/data/labels' + , cb + + getPropertyKeys: (cb) -> + # This endpoint returns the array of property keys directly: + # http://neo4j.com/docs/stable/rest-api-property-values.html#rest-api-list-all-property-keys + # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + # TODO: Would it be better for us to handle other non-200 responses too? + @http + method: 'GET' + path: '/db/data/propertykeys' + , cb + + getRelationshipTypes: (cb) -> + # This endpoint returns the array of relationship types directly: + # http://neo4j.com/docs/stable/rest-api-relationship-types.html#rest-api-get-relationship-types + # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + # TODO: Would it be better for us to handle other non-200 responses too? + @http + method: 'GET' + path: '/db/data/relationship/types' + , cb + + # TODO: Indexes + # TODO: Constraints + # TODO: Legacy indexing + + ## HELPERS # diff --git a/test-new/schema._coffee b/test-new/schema._coffee new file mode 100644 index 0000000..68260a1 --- /dev/null +++ b/test-new/schema._coffee @@ -0,0 +1,51 @@ +# +# Tests for schema management, e.g. retrieving labels, property keys, and +# relationship types. Could also encompass constraints and (schema) indexes, +# but that could be its own test suite too if it goes deeper. +# + +{expect} = require 'chai' +fixtures = require './fixtures' +neo4j = require '../' + + +## SHARED STATE + +{DB, TEST_LABEL, TEST_REL_TYPE} = fixtures + +[TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] + + +## TESTS + +describe 'Schema', -> + + it '(create test graph)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ + + it 'should support listing all labels', (_) -> + labels = DB.getLabels _ + + expect(labels).to.be.an 'array' + expect(labels).to.not.be.empty() + expect(labels).to.contain TEST_LABEL + + it 'should support listing all property keys', (_) -> + keys = DB.getPropertyKeys _ + + expect(keys).to.be.an 'array' + expect(keys).to.not.be.empty() + + for key of TEST_NODE_A.properties + expect(keys).to.contain key + + it 'should support listing all relationship types', (_) -> + types = DB.getRelationshipTypes _ + + expect(types).to.be.an 'array' + expect(types).to.not.be.empty() + expect(types).to.contain TEST_REL_TYPE + + it '(delete test graph)', (_) -> + fixtures.deleteTestGraph module, _ From 347629ca43171930b49cc14c40ef241bdfbf706e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Feb 2015 00:16:23 -0500 Subject: [PATCH 030/121] v2 / Misc: bump alpha version to 3. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 285a860..5a98093 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-alpha2", + "version": "2.0.0-alpha3", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From 49ea5cc3570380e27d85c48fe1955d854b62bc89 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Feb 2015 02:01:45 -0500 Subject: [PATCH 031/121] v2 / Docs: remove Codo. Unfortunately it seems broken at the moment: https://github.com/coffeedoc/codo/issues/194 https://github.com/coffeedoc/codo/issues/195 And the auto-webhook site is down too: https://github.com/coffeedoc/coffeedoc.info/issues/9 Given these issues, going to just write docs manually. --- .codoopts | 4 ---- .gitignore | 2 -- package.json | 2 -- 3 files changed, 8 deletions(-) delete mode 100644 .codoopts diff --git a/.codoopts b/.codoopts deleted file mode 100644 index 3e0d9af..0000000 --- a/.codoopts +++ /dev/null @@ -1,4 +0,0 @@ ---name 'Node-Neo4j API Documentation' ---title 'Node-Neo4j API Documentation' ---readme API.md -./lib-old diff --git a/.gitignore b/.gitignore index 386e0bb..de5f125 100644 --- a/.gitignore +++ b/.gitignore @@ -3,7 +3,5 @@ node_modules/ npm-debug.log -/doc/ - /neo4j-* /neo4j diff --git a/package.json b/package.json index 5a98093..517f675 100644 --- a/package.json +++ b/package.json @@ -14,7 +14,6 @@ }, "devDependencies": { "chai": "^1.9.2", - "codo": "^2.0.9", "coffee-script": "1.8.x", "mocha": "^2.0.1", "streamline": "^0.10.16" @@ -25,7 +24,6 @@ "scripts": { "build": "coffee -m -c lib-new/", "clean": "rm -f lib-new/*.{js,map}", - "codo": "codo && codo --server", "prepublish": "npm run build", "postpublish": "npm run clean", "test": "mocha test-new" From e4c030a66b05687c153d03c8b01ec3442b4bb379 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Feb 2015 03:12:37 -0500 Subject: [PATCH 032/121] v2 / Docs: WIP readme --- README.md | 92 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 60 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 2f16be7..a7523d4 100644 --- a/README.md +++ b/README.md @@ -1,58 +1,86 @@ + -This is a client library for accessing [Neo4j][], a graph database, from -[Node.js][]. It uses Neo4j's [REST API][neo4j-rest-api], and supports -Neo4j 1.5 through Neo4j 2.1. +# Node-Neo4j npm version -(Note that new 2.0 features like labels and constraints are only accessible -through Cypher for now -- but Cypher is the recommended interface for Neo4j -anyway. This driver might change to wrap Cypher entirely.) +This is a Node.js driver for [Neo4j](http://neo4j.com/), a graph database. -**Update: [node-neo4j v2][] is under development and almost finished!** -[Read the full spec here][v2 spec], and follow the progress in [pull #145][]. -If you're comfortable using pre-release code, alpha versions are available on -npm; we at [FiftyThree][] are running them in production. =) +**This driver has undergone a complete rewrite for Neo4j v2.** +It now *only* supports Neo4j 2.x — but it supports it really well. +(If you're still on Neo4j 1.x, you can still use +[node-neo4j v1](https://github.com/thingdom/node-neo4j/tree/v1).) -[node-neo4j v2]: https://github.com/thingdom/node-neo4j/tree/v2#readme -[v2 spec]: https://github.com/thingdom/node-neo4j/blob/v2/API_v2.md -[pull #145]: https://github.com/thingdom/node-neo4j/pull/145 -[FiftyThree]: http://www.fiftythree.com/ + + + ## Installation - npm install neo4j@1.x --save +```sh +npm install neo4j --save +``` ## Usage -To start, create a new instance of the `GraphDatabase` class pointing to your -Neo4j instance: - ```js var neo4j = require('neo4j'); var db = new neo4j.GraphDatabase('http://localhost:7474'); -``` - -Node.js is asynchronous, which means this library is too: most functions take -callbacks and return immediately, with the callbacks being invoked when the -corresponding HTTP requests and responses finish. -Here's a simple example: - -```js -var node = db.createNode({hello: 'world'}); // instantaneous, but... -node.save(function (err, node) { // ...this is what actually persists. - if (err) { - console.error('Error saving new node to database:', err); +db.cypher({ + query: 'MATCH (u:User {email: {email}}) RETURN u', + params: { + email: 'alice@example.com', + }, +}, function (err, results) { + if (err) throw err; + var result = results[0]; + if (!result) { + console.log('No user found.'); } else { - console.log('Node saved to database with id:', node.id); + var user = result['u']; + console.log(JSON.stringify(user, null, 4)); } }); ``` +Yields e.g.: + +```json +{ + "_id": 12345678, + "labels": [ + "User", + "Admin" + ], + "properties": { + "name": "Alice Smith", + "email": "alice@example.com", + "emailVerified": true, + "passwordHash": "..." + } +} +``` + + + +Node.js is asynchronous, which means this library is too: most functions take +callbacks and return immediately, with the callbacks being invoked when the +corresponding HTTP requests and responses finish. + Because async flow in Node.js can be quite tricky to handle, we strongly recommend using a flow control tool or library to help. Our personal favorite is [Streamline.js][], but other popular choices are From 25d65236be94c7026ad213d73e289ad09413c4cf Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 16 Feb 2015 21:38:02 -0500 Subject: [PATCH 033/121] v2 / Errors: include Neo4j stack on database errors! --- lib-new/errors.coffee | 35 ++++++++++++++++++++++++++++------- test-new/transactions._coffee | 35 ++++++++++++++++++++++++++++------- 2 files changed, 56 insertions(+), 14 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 3991e98..3d0f785 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -45,18 +45,39 @@ class @Error extends Error @_fromTransaction: (obj) -> # http://neo4j.com/docs/stable/rest-api-transactional.html#rest-api-handling-errors # http://neo4j.com/docs/stable/status-codes.html - {code, message} = obj + {code, message, stackTrace} = obj [neo, classification, category, title] = code.split '.' ErrorClass = exports[classification] # e.g. DatabaseError - message = "[#{category}.#{title}] #{message or '(no message)'}" - # TODO: Some errors (always DatabaseErrors?) can also apparently have a - # `stack` property with the Java stack trace. Should we include it in - # our own message/stack, in the DatabaseError case at least? - # (This'd be analagous to including the body for 5xx responses above.) + # Prefix all messages with the classification details: + fullMessage = "[#{category}.#{title}] " + + # If this is a database error with a Java stack trace from Neo4j, + # include that stack, for bug reporting to the Neo4j team. + # Also include the stack if there's no summary message. + # TODO: Should we make it configurable to always include it? + # NOTE: The stack seems to be returned as a string, not an array. + if stackTrace and (classification is 'DatabaseError' or not message) + # It seems that this stack trace includes the summary message, + # but checking just in case it doesn't, and adding it if so. + if message and (stackTrace.indexOf message) is -1 + stackTrace = "#{message}: #{stackTrace}" + + # Stack traces can include "Caused by" lines which aren't indented, + # and indented lines use tabs. Normalize to 4 spaces, and indent + # everything one extra level, to differentiate from Node.js lines. + stackTrace = stackTrace + .replace /\t/g, ' ' + .replace /\n/g, '\n ' + + fullMessage += stackTrace + + # Otherwise, e.g. for client errors, omit any stack; just the message: + else + fullMessage += message - new ErrorClass message, obj + new ErrorClass fullMessage, obj # TODO: Helper to rethrow native/inner errors? Not sure if we need one. diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index c219e08..6e359ae 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -21,18 +21,38 @@ neo4j = require '../' # # Asserts that the given object is an instance of the proper Neo4j Error # subclass, representing the given transactional Neo4j error info. -# TODO: Consider consolidating with a similar helper in the `http` test suite. +# TODO: De-duplicate with same helper in Cypher test suite! # expectError = (err, classification, category, title, message) -> expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError expect(err.name).to.equal "neo4j.#{classification}" - expect(err.message).to.equal "[#{category}.#{title}] #{message}" + + # If the actual error message is multi-line, it includes the Neo4j stack + # trace; test that in a simple way by just checking the first line of the + # trace (subsequent lines can be different, e.g. "Caused by"), but also test + # that the first line of the message matches the expected message: + expect(err.message).to.be.a 'string' + [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' + expect(errMessageLine1).to.equal "[#{category}.#{title}] #{message}" + expect(errMessageLine2).to.match /// + ^ \s+ at\ [^(]+ \( [^)]+ [.]java:\d+ \) + /// if errMessageLine2 + + expect(err.stack).to.be.a 'string' expect(err.stack).to.contain '\n' - expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + expect(err.stack).to.contain "#{err.name}: #{err.message}" + [errStackLine1, ...] = err.stack.split '\n' + expect(errStackLine1).to.equal "#{err.name}: #{errMessageLine1}" + expect(err.neo4j).to.be.an 'object' - expect(err.neo4j).to.contain - code: "Neo.#{classification}.#{category}.#{title}" - message: message + expect(err.neo4j.code).to.equal "Neo.#{classification}.#{category}.#{title}" + # If the actual error message was multi-line, that means it was the Neo4j + # stack trace, which can include a larger message than the returned one. + if errMessageLine2 + expect(err.neo4j.message).to.be.a 'string' + expect(message).to.contain err.neo4j.message + else + expect(err.neo4j.message).to.equal message ## TESTS @@ -469,7 +489,8 @@ describe 'Transactions', -> try expect(err).to.exist() expectError err, 'DatabaseError', 'Transaction', - 'CouldNotCommit', 'javax.transaction.RollbackException: + 'CouldNotCommit', 'java.lang.RuntimeException: + javax.transaction.RollbackException: Failed to commit, transaction rolled back' catch assertionErr return cont assertionErr From 7e663ec8180fe72d8b814f369b7ca3cfe4cec096 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 18 Feb 2015 20:59:07 -0500 Subject: [PATCH 034/121] v2 / Misc: bump alpha version to 4. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 517f675..3205dc0 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-alpha3", + "version": "2.0.0-alpha4", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From e708bf4ab6b380fe4da00463a3d8a59e49da4a3e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 16 Feb 2015 21:43:04 -0500 Subject: [PATCH 035/121] v2 / Misc: rename index.coffee to exports.coffee. To pave the way for an Index class. --- lib-new/{index.coffee => exports.coffee} | 0 package.json | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) rename lib-new/{index.coffee => exports.coffee} (100%) diff --git a/lib-new/index.coffee b/lib-new/exports.coffee similarity index 100% rename from lib-new/index.coffee rename to lib-new/exports.coffee diff --git a/package.json b/package.json index 3205dc0..ffe175b 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,7 @@ "Daniel Gasienica ", "Sergio Haro " ], - "main": "./lib-new", + "main": "./lib-new/exports", "dependencies": { "request": "^2.27.0", "underscore": "1.7.x" From 82adacee56fc3b78d496f12d3700f13cf905f16f Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 23 Feb 2015 22:54:11 -0500 Subject: [PATCH 036/121] v2 / Tests: update Travis to Neo4j 2.1.7. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index d079e2a..c7f1293 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,7 +10,7 @@ jdk: env: # test across multiple versions of Neo4j: - NEO4J_VERSION="2.2.0-M03" - - NEO4J_VERSION="2.1.6" + - NEO4J_VERSION="2.1.7" - NEO4J_VERSION="2.0.4" matrix: From 056d101c56b3dd4823337e946e52aaed1a4aafbe Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 25 Feb 2015 20:10:11 -0500 Subject: [PATCH 037/121] v2 / Core: work around Neo4j 2.1.7 hyperlink bug. --- lib-new/utils.coffee | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib-new/utils.coffee b/lib-new/utils.coffee index ed75112..23e6dd6 100644 --- a/lib-new/utils.coffee +++ b/lib-new/utils.coffee @@ -7,6 +7,9 @@ # e.g. nodes, relationships, even transactions. # @parseId = (url) -> - match = url.match /// /db/data/\w+/(\d+)($|/) /// + # NOTE: Neo4j 2.1.7 shipped a bug with hypermedia links returned from the + # transactional endpoint, so we have to account for that: + # https://github.com/neo4j/neo4j/issues/4076 + match = url.match /// (?:commit|/)db/data/\w+/(\d+)($|/) /// return null if not match return parseInt match[1], 10 From a3b84258505301238324a3b707e2f14fb4ca2757 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 00:58:08 -0500 Subject: [PATCH 038/121] v2 / Tests: trigger standalone DatabaseError. --- test-new/transactions._coffee | 36 ++++++++++++----------------------- 1 file changed, 12 insertions(+), 24 deletions(-) diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 6e359ae..16017fc 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -35,7 +35,7 @@ expectError = (err, classification, category, title, message) -> [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' expect(errMessageLine1).to.equal "[#{category}.#{title}] #{message}" expect(errMessageLine2).to.match /// - ^ \s+ at\ [^(]+ \( [^)]+ [.]java:\d+ \) + ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) /// if errMessageLine2 expect(err.stack).to.be.a 'string' @@ -465,33 +465,21 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'fatal database errors' expect(nodeA.properties.i).to.equal 1 - # The only way I know how to trigger a database error is to trigger a - # client error, and then *separately* attempt to commit the transaction. - # TODO: Is there any better way? - try - tx.cypher - query: ''' - START nodeA = node({idA}) - SET nodeA.i = 2 - RETURN {foo} - ''' - params: - idA: TEST_NODE_A._id - , _ - catch err - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - + # HACK: Depending on a known bug to trigger a DatabaseError; + # that makes this test brittle, since the bug could get fixed! + # https://github.com/neo4j/neo4j/issues/3870#issuecomment-76650113 # For precision, implementing this step without Streamline. do (cont=_) => - tx.commit (err) => + tx.cypher + query: 'CREATE (n {props})' + params: + props: {foo: null} + , (err, results) => try expect(err).to.exist() - expectError err, 'DatabaseError', 'Transaction', - 'CouldNotCommit', 'java.lang.RuntimeException: - javax.transaction.RollbackException: - Failed to commit, transaction rolled back' + expectError err, + 'DatabaseError', 'Statement', 'ExecutionFailure', + 'scala.MatchError: (foo,null) (of class scala.Tuple2)' catch assertionErr return cont assertionErr cont() From 4f8704fb450329490b74fb8f56672b45e9ad3681 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 00:58:24 -0500 Subject: [PATCH 039/121] v2 / Tests: add transactional "first query" tests. --- test-new/transactions._coffee | 63 +++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 16017fc..e6d4761 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -503,6 +503,69 @@ describe 'Transactions', -> # TODO: Is there any way to trigger and test transient errors? + it 'should properly handle non-fatal errors on the first query', (_) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher 'RETURN {foo}', (err, results) => + try + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'open' + + it 'should properly handle fatal client errors + on an auto-commit first query', (_) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher + query: 'RETURN {foo}' + commit: true + , (err, results) => + try + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'rolled back' + + it 'should properly handle fatal database errors on the first query', (_) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + # HACK: Depending on a known bug to trigger a DatabaseError; + # that makes this test brittle, since the bug could get fixed! + # https://github.com/neo4j/neo4j/issues/3870#issuecomment-76650113 + # For precision, implementing this step without Streamline. + do (cont=_) => + tx.cypher + query: 'CREATE (n {props})' + params: + props: {foo: null} + , (err) => + try + expect(err).to.exist() + expectError err, + 'DatabaseError', 'Statement', 'ExecutionFailure', + 'scala.MatchError: (foo,null) (of class scala.Tuple2)' + catch assertionErr + return cont assertionErr + cont() + + expect(tx.state).to.equal 'rolled back' + it 'should properly handle errors with batching', (_) -> tx = DB.beginTransaction() From 5f5e5901c5b539dcb0cd6318a4e51c3b1288d1a6 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 00:58:48 -0500 Subject: [PATCH 040/121] v2 / Tests: test transaction commit before any queries. --- test-new/transactions._coffee | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index e6d4761..e32429a 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -165,6 +165,13 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'committing' + it 'should support committing before any queries', (_) -> + tx = DB.beginTransaction() + expect(tx.state).to.equal 'open' + + tx.commit _ + expect(tx.state).to.equal 'committed' + it 'should support auto-committing', (_) -> tx = DB.beginTransaction() @@ -253,7 +260,7 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.not.equal 'rolling back' - it 'should support rolling back before any commits', (_) -> + it 'should support rolling back before any queries', (_) -> tx = DB.beginTransaction() expect(tx.state).to.equal 'open' @@ -554,7 +561,7 @@ describe 'Transactions', -> query: 'CREATE (n {props})' params: props: {foo: null} - , (err) => + , (err, results) => try expect(err).to.exist() expectError err, From 2e6d5cf69c8c2d95b75742a4f44ce0faddb00c1c Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 01:06:28 -0500 Subject: [PATCH 041/121] v2 / Transactions: fix/support committing before created. --- lib-new/GraphDatabase.coffee | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 8d1427a..0f00c97 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -88,21 +88,18 @@ module.exports = class GraphDatabase if not _tx and rollback throw new Error 'Illegal state: rolling back without a transaction!' - if not _tx?._id and rollback - # No query has been made within transaction yet, so this transaction - # doesn't even exist yet from Neo4j's POV; nothing to do. - cb null, null - return - if commit and rollback throw new Error 'Illegal state: both committing and rolling back!' + if rollback and (query or queries) + throw new Error 'Illegal state: rolling back with query/queries!' + if not _tx and commit is false throw new TypeError 'Can’t refuse to commit without a transaction! To begin a new transaction without committing, call `db.beginTransaction()`, and then call `cypher` on that.' - if not _tx and not query and not queries + if not _tx and not (query or queries) throw new TypeError 'Query or queries required' if query and queries @@ -117,6 +114,14 @@ module.exports = class GraphDatabase throw new TypeError 'When batching multiple queries, `lean` must be specified with each query, not globally.' + if (commit or rollback) and not (query or queries) and not _tx._id + # (Note that we've already required query or queries if no + # transaction present, so this means a transaction is present.) + # This transaction hasn't even been created yet from Neo4j's POV + # (because transactions are created lazily), so nothing to do. + cb null, null + return + method = 'POST' method = 'DELETE' if rollback From af90dcc2e23eb1716524de56eaaf55b76312d1d7 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 01:26:47 -0500 Subject: [PATCH 042/121] v2 / Errors: improve message/stack logging. --- lib-new/errors.coffee | 6 +++--- test-new/cypher._coffee | 10 ++++++---- test-new/http._coffee | 4 ++-- test-new/transactions._coffee | 7 +++++-- 4 files changed, 16 insertions(+), 11 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 3d0f785..38ed513 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -22,7 +22,7 @@ class @Error extends Error ErrorType = if statusCode >= 500 then 'Database' else 'Client' ErrorClass = exports["#{ErrorType}Error"] - message = "[#{statusCode}] " + message = "#{statusCode} " logBody = statusCode >= 500 # TODO: Config to always log body? if body?.exception @@ -50,8 +50,8 @@ class @Error extends Error ErrorClass = exports[classification] # e.g. DatabaseError - # Prefix all messages with the classification details: - fullMessage = "[#{category}.#{title}] " + # Prefix all messages with the full semantic code, for at-a-glance-ness: + fullMessage = "[#{code}] " # If this is a database error with a Java stack trace from Neo4j, # include that stack, for bug reporting to the Neo4j team. diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 0a34c6c..b5f26c2 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -23,15 +23,17 @@ neo4j = require '../' # TODO: Consider consolidating with a similar helper in the `http` test suite. # expectError = (err, classification, category, title, message) -> + code = "Neo.#{classification}.#{category}.#{title}" + expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError expect(err.name).to.equal "neo4j.#{classification}" - expect(err.message).to.equal "[#{category}.#{title}] #{message}" + expect(err.message).to.equal "[#{code}] #{message}" + expect(err.stack).to.contain '\n' expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + expect(err.neo4j).to.be.an 'object' - expect(err.neo4j).to.eql - code: "Neo.#{classification}.#{category}.#{title}" - message: message + expect(err.neo4j).to.eql {code, message} ## TESTS diff --git a/test-new/http._coffee b/test-new/http._coffee index 0e180d3..ec0013f 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -77,7 +77,7 @@ describe 'GraphDatabase::http', -> expect(body).to.not.exist() expectError err, neo4j.ClientError, - '[405] Method Not Allowed response for POST /' + '405 Method Not Allowed response for POST /' expect(err.neo4j).to.be.empty() catch assertionErr @@ -94,7 +94,7 @@ describe 'GraphDatabase::http', -> expect(err).to.exist() expect(body).to.not.exist() - expectError err, neo4j.ClientError, '[404] [NodeNotFoundException] + expectError err, neo4j.ClientError, '404 [NodeNotFoundException] Cannot find node with id [-1] in database.' expect(err.neo4j).to.be.an 'object' diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index e32429a..ae53d45 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -24,6 +24,8 @@ neo4j = require '../' # TODO: De-duplicate with same helper in Cypher test suite! # expectError = (err, classification, category, title, message) -> + code = "Neo.#{classification}.#{category}.#{title}" + expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError expect(err.name).to.equal "neo4j.#{classification}" @@ -33,7 +35,7 @@ expectError = (err, classification, category, title, message) -> # that the first line of the message matches the expected message: expect(err.message).to.be.a 'string' [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' - expect(errMessageLine1).to.equal "[#{category}.#{title}] #{message}" + expect(errMessageLine1).to.equal "[#{code}] #{message}" expect(errMessageLine2).to.match /// ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) /// if errMessageLine2 @@ -45,7 +47,8 @@ expectError = (err, classification, category, title, message) -> expect(errStackLine1).to.equal "#{err.name}: #{errMessageLine1}" expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.code).to.equal "Neo.#{classification}.#{category}.#{title}" + expect(err.neo4j.code).to.equal code + # If the actual error message was multi-line, that means it was the Neo4j # stack trace, which can include a larger message than the returned one. if errMessageLine2 From 02d795f3dbe5719f700a77ccede01357917e5c32 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 01:28:31 -0500 Subject: [PATCH 043/121] v2 / Misc: bump alpha version to 5. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index ffe175b..cc3975f 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-alpha4", + "version": "2.0.0-alpha5", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From 9d9fd7b17a5b2e2ddd5d8069bbf71200606d079e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 19:46:27 -0500 Subject: [PATCH 044/121] v2 / Tests: standardize .travis.yml on 2-space indent. --- .travis.yml | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/.travis.yml b/.travis.yml index c7f1293..d47c3f4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,30 +1,30 @@ language: node_js node_js: - - "0.12" - - "0.10" + - "0.12" + - "0.10" jdk: - - oraclejdk7 # needed for local Neo4j 2.0+ + - oraclejdk7 # needed for local Neo4j 2.0+ env: - # test across multiple versions of Neo4j: - - NEO4J_VERSION="2.2.0-M03" - - NEO4J_VERSION="2.1.7" - - NEO4J_VERSION="2.0.4" + # test across multiple versions of Neo4j: + - NEO4J_VERSION="2.2.0-M03" + - NEO4J_VERSION="2.1.7" + - NEO4J_VERSION="2.0.4" matrix: - # but we may want to allow our tests to fail against future, unstable - # versions of Neo4j. E.g. 2.2 introduces auth, and sets it by default. - # TODO: remove this once we've added auth support and fixed it for 2.2. - allow_failures: - - env: NEO4J_VERSION="2.2.0-M03" + # but we may want to allow our tests to fail against future, unstable + # versions of Neo4j. E.g. 2.2 introduces auth, and sets it by default. + # TODO: remove this once we've added auth support and fixed it for 2.2. + allow_failures: + - env: NEO4J_VERSION="2.2.0-M03" before_install: - # install Neo4j locally: - - wget dist.neo4j.org/neo4j-community-$NEO4J_VERSION-unix.tar.gz - - tar -xzf neo4j-community-$NEO4J_VERSION-unix.tar.gz - - neo4j-community-$NEO4J_VERSION/bin/neo4j start + # install Neo4j locally: + - wget dist.neo4j.org/neo4j-community-$NEO4J_VERSION-unix.tar.gz + - tar -xzf neo4j-community-$NEO4J_VERSION-unix.tar.gz + - neo4j-community-$NEO4J_VERSION/bin/neo4j start # don't CI feature branches, but note that this *does* CI PR merge commits -- # including before they're made! =) From dddac38d754db4c58fac157b8030b65cd0c4fc46 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 19:51:46 -0500 Subject: [PATCH 045/121] v2 / Tests: update .travis.yml to Neo4j 2.2 RC1 + auth. --- .travis.yml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index d47c3f4..3850a5f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -9,16 +9,15 @@ jdk: env: # test across multiple versions of Neo4j: - - NEO4J_VERSION="2.2.0-M03" + - NEO4J_VERSION="2.2.0-RC01" - NEO4J_VERSION="2.1.7" - NEO4J_VERSION="2.0.4" matrix: - # but we may want to allow our tests to fail against future, unstable - # versions of Neo4j. E.g. 2.2 introduces auth, and sets it by default. - # TODO: remove this once we've added auth support and fixed it for 2.2. + # but we may want to allow our tests to fail against *some* Neo4j versions, + # e.g. due to unstability, bugs, or breaking changes for our test code. allow_failures: - - env: NEO4J_VERSION="2.2.0-M03" + - env: NEO4J_VERSION="2.0.4" # seems to have transaction bugs before_install: # install Neo4j locally: From 3af5f5134885c71f37a917176a8330a1e68b22f1 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 22:01:12 -0500 Subject: [PATCH 046/121] v2 / Auth: add, implement, test `auth` property! --- API_v2.md | 18 +++++-- lib-new/GraphDatabase.coffee | 32 ++++++++++++- test-new/constructor._coffee | 92 ++++++++++++++++++++++++++++++++++-- 3 files changed, 133 insertions(+), 9 deletions(-) diff --git a/API_v2.md b/API_v2.md index dc928ac..d1d109b 100644 --- a/API_v2.md +++ b/API_v2.md @@ -44,15 +44,25 @@ var neo4j = require('neo4j'); var db = new neo4j.GraphDatabase({ url: 'http://localhost:7474', + auth: null, // optional; see below for more details headers: {}, // optional defaults, e.g. User-Agent - proxy: '', // optional URL + proxy: null, // optional URL agent: null, // optional http.Agent instance, for custom socket pooling }); ``` -An upcoming version of Neo4j will likely add native authentication. -We already support HTTP Basic Auth in the URL, but we may then need to add -ways to manage the auth (e.g. generate and reset tokens). +To specify auth credentials, the username and password can be provided either +directly in the URL, e.g. `'http://user:pass@localhost:7474'`, +or via an `auth` option, which can either be a `'username:password'` string +or a `{username, password}` object. The auth option takes precedence. + +If credentials are given in any form, they will be normalized to a +`{username, password}` object and set as the `auth` property on the +constructed `GraphDatabase` instance. +In addition, the `url` property will have its credentials cleared if an `auth` +option was provided as well. +An empty string/object `auth` can be provided to clear auth in this way; +the `auth` property will be normalized to `null` in this case. The current v1 of the driver is hypermedia-driven, so it discovers the `/db/data` endpoint. We may hardcode that in v2 for efficiency and simplicity, diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 0f00c97..29ff83c 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -20,11 +20,21 @@ module.exports = class GraphDatabase if typeof opts is 'string' opts = {url: opts} - {@url, @headers, @proxy, @agent} = opts + {@url, @auth, @headers, @proxy, @agent} = opts if not @url throw new TypeError 'URL to Neo4j required' + # Process auth, whether through option or URL creds or both. + # Option takes precedence, and we clear the URL creds if option given. + uri = URL.parse @url + if uri.auth and @auth? + delete uri.auth + @url = URL.format uri + + # We also normalize any given auth to an object or null: + @auth = _normalizeAuth @auth ? uri.auth + # TODO: Do we want to special-case User-Agent? Blacklist X-Stream? @headers or= {} $(@headers).defaults @constructor::headers @@ -47,6 +57,7 @@ module.exports = class GraphDatabase method: method url: URL.resolve @url, path proxy: @proxy + auth: @auth headers: $(headers).defaults @headers agent: @agent json: body ? true @@ -295,6 +306,25 @@ module.exports = class GraphDatabase ## HELPERS +# +# Normalizes the given auth value, which can be a 'username:password' string +# or a {username, password} object, to an object or null always. +# +_normalizeAuth = (auth) -> + # Support empty string for no auth: + return null if not auth + + # Parse string if given, being robust to colons in the password: + if typeof auth is 'string' + [username, passwordParts...] = auth.split ':' + password = passwordParts.join ':' + auth = {username, password} + + # Support empty object for no auth also: + return null if (Object.keys auth).length is 0 + + auth + # # Deep inspects the given object -- which could be a simple primitive, a map, # an array with arbitrary other objects, etc. -- and transforms any objects that diff --git a/test-new/constructor._coffee b/test-new/constructor._coffee index 34d4f4d..b2ffa63 100644 --- a/test-new/constructor._coffee +++ b/test-new/constructor._coffee @@ -9,14 +9,17 @@ $ = require 'underscore' ## CONSTANTS -URL = 'http://foo:bar@baz:1234' -PROXY = 'http://lorem.ipsum' +URL = 'https://example.com:1234' +PROXY = 'https://some.proxy:5678' HEADERS = 'x-foo': 'bar-baz' 'x-lorem': 'ipsum' # TODO: Test overlap with default headers? # TODO: Test custom User-Agent behavior, or blacklist X-Stream? +USERNAME = 'alice' +PASSWORD = 'p4ssw0rd' + ## HELPERS @@ -50,6 +53,12 @@ expectHeaders = (db, headers) -> for key, val of db.headers expect(val).to.equal headers[key] or defaultHeaders[key] +expectAuth = (db, username, password) -> + expect(db.auth).to.eql {username, password} + +expectNoAuth = (db) -> + expect(db.auth).to.not.exist() + ## TESTS @@ -63,17 +72,92 @@ describe 'GraphDatabase::constructor', -> expectDatabase db, URL, PROXY expectHeaders db, HEADERS + expectNoAuth db it 'should support just URL string', -> db = new GraphDatabase URL expectDatabase db, URL expectHeaders db, {} + expectNoAuth db it 'should throw if no URL given', -> fn = -> new GraphDatabase() - expect(fn).to.throw TypeError + expect(fn).to.throw TypeError, /URL to Neo4j required/ # Also try giving an options argument, just with no URL: fn = -> new GraphDatabase {proxy: PROXY} - expect(fn).to.throw TypeError + expect(fn).to.throw TypeError, /URL to Neo4j required/ + + it 'should support and parse auth in URL', -> + url = "https://#{USERNAME}:#{PASSWORD}@auth.test:9876" + db = new GraphDatabase url + + expectDatabase db, url + expectAuth db, USERNAME, PASSWORD + + it 'should support and parse auth as separate string option', -> + db = new GraphDatabase + url: URL + auth: "#{USERNAME}:#{PASSWORD}" + + expectDatabase db, URL + expectAuth db, USERNAME, PASSWORD + + it 'should support and parse auth as separate object option', -> + db = new GraphDatabase + url: URL + auth: + username: USERNAME + password: PASSWORD + + expectDatabase db, URL + expectAuth db, USERNAME, PASSWORD + + it 'should prefer separate auth option over auth in the URL + (and should clear auth in URL then)', -> + host = 'auth.test:9876' + wrong1 = "#{Math.random()}"[2..] + wrong2 = "#{Math.random()}"[2..] + + db = new GraphDatabase + url: "https://#{wrong1}:#{wrong2}@#{host}" + auth: "#{USERNAME}:#{PASSWORD}" + + # NOTE: The constructor adds a trailing slash, but that's okay. + expectDatabase db, "https://#{host}/" + expectAuth db, USERNAME, PASSWORD + + it 'should support clearing auth via empty string option', -> + host = "auth.test:9876" + url = "https://#{USERNAME}:#{PASSWORD}@#{host}" + + db = new GraphDatabase + url: url + auth: '' + + # NOTE: The constructor adds a trailing slash, but that's okay. + expectDatabase db, "https://#{host}/" + expectNoAuth db + + it 'should support clearing auth via empty object option', -> + host = "auth.test:9876" + url = "https://#{USERNAME}:#{PASSWORD}@#{host}" + + db = new GraphDatabase + url: url + auth: {} + + # NOTE: The constructor adds a trailing slash, but that's okay. + expectDatabase db, "https://#{host}/" + expectNoAuth db + + it 'should be robust to colons in the password with string option', -> + password = "#{PASSWORD}:#{PASSWORD}:#{PASSWORD}" + + db = new GraphDatabase + url: URL + auth: "#{USERNAME}:#{password}" + + expectDatabase db, URL + expectAuth db, USERNAME, password From 9c998c6c66d53539a7b4abc7adbe72af7e3f02e9 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 23:46:20 -0500 Subject: [PATCH 047/121] v2 / Auth: add, implement, test password management! --- API_v2.md | 19 +++++ lib-new/GraphDatabase.coffee | 51 +++++++++++++ test-new/_auth._coffee | 122 ++++++++++++++++++++++++++++++++ test-new/fixtures/index._coffee | 7 +- 4 files changed, 197 insertions(+), 2 deletions(-) create mode 100644 test-new/_auth._coffee diff --git a/API_v2.md b/API_v2.md index d1d109b..2015070 100644 --- a/API_v2.md +++ b/API_v2.md @@ -4,6 +4,7 @@ This is a rough, work-in-progress redesign of the node-neo4j API. - [General](#general) - [Core](#core) +- [Auth](#auth) - [HTTP](#http) - [Objects](#objects) - [Cypher](#cypher) @@ -69,6 +70,24 @@ The current v1 of the driver is hypermedia-driven, so it discovers the but if we do, do we need to make that customizable/overridable too? +## Auth + +**Let me manage database authentication.** + +Note that this section only matters for *managing* auth, not *specifying* it. + +```js +function cbBool(err, bool) {} +function cbDone(err) {} + +db.checkPasswordChangeNeeded(cbBool); +db.changePassword({password}, cbDone); +``` + +For convenience, changing the password will automatically update the `auth` +property on the `GraphDatabase` instance, so that subsequent requests will work. + + ## HTTP **Let me make arbitrary HTTP requests to the REST API.** diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 29ff83c..f0c1d75 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -11,6 +11,9 @@ URL = require 'url' module.exports = class GraphDatabase + + ## CORE + # Default HTTP headers: headers: 'User-Agent': "node-neo4j/#{lib.version}" @@ -39,6 +42,9 @@ module.exports = class GraphDatabase @headers or= {} $(@headers).defaults @constructor::headers + + ## HTTP + http: (opts={}, cb) -> if typeof opts is 'string' opts = {path: opts} @@ -87,6 +93,51 @@ module.exports = class GraphDatabase req.start() req.req + + ## AUTH + + checkPasswordChangeNeeded: (cb) -> + if not @auth?.username + throw new TypeError 'No `auth` specified in constructor!' + + @http + method: 'GET' + path: "/user/#{encodeURIComponent @auth.username}" + , (err, user) -> + if err + return cb err + + cb null, user.password_change_required + + changePassword: (opts={}, cb) -> + if typeof opts is 'string' + opts = {password: opts} + + {password} = opts + + if not @auth?.username + throw new TypeError 'No `auth` specified in constructor!' + + if not password + throw new TypeError 'Password required' + + @http + method: 'POST' + path: "/user/#{encodeURIComponent @auth.username}/password" + body: {password} + , (err) => + if err + return cb err + + # Since successful, update our saved state for subsequent requests: + @auth.password = password + + # Void method: + cb null + + + ## CYPHER + cypher: (opts={}, cb, _tx) -> if typeof opts is 'string' opts = {query: opts} diff --git a/test-new/_auth._coffee b/test-new/_auth._coffee new file mode 100644 index 0000000..c8cc931 --- /dev/null +++ b/test-new/_auth._coffee @@ -0,0 +1,122 @@ +# +# Tests for basic auth management, e.g. the ability to change user passwords. +# +# IMPORTANT: Against a fresh Neo4j 2.2 database (which requires both auth and +# an initial password change by default), this test must be the first to run, +# in order for other tests not to fail. Hence the underscore-prefixed filename. +# (The original password is restored at the end of this suite.) +# +# NOTE: Since auth (a) can be disabled, and (b) isn't supported by Neo4j <2.2, +# this suite first checks if auth is enabled, and *only* runs if it is. +# If auth is disabled or not present, every test here will be skipped. +# + +crypto = require 'crypto' +{expect} = require 'chai' +fixtures = require './fixtures' +neo4j = require '../' + + +## SHARED STATE + +{DB} = fixtures + +ORIGINAL_PASSWORD = DB.auth?.password +RANDOM_PASSWORD = crypto.randomBytes(16).toString('base64') + +SUITE = null + + +## HELPERS + +disable = (reason) -> + console.warn "#{reason}; not running auth tests." + + # HACK: Perhaps relying on Mocha's internals to achieve skip all: + for test, i in SUITE.tests + continue if i is 0 # This is our "check" test + test.pending = true + + # TODO: It'd be nice if we could support bailing on *all* suites, + # in case of an auth error, e.g. bad credentials. + +tryGetDbVersion = (_) -> + try + fixtures.queryDbVersion _ + fixtures.DB_VERSION_NUM + catch err + NaN # since a boolean comparison of NaN with any number is false + + +## TESTS + +describe 'Auth', -> + + SUITE = @ + + it '(check if auth is enabled)', (_) -> + if not ORIGINAL_PASSWORD + return disable 'Auth creds unspecified' + + # Querying user status (what this check method does) fails both when + # auth is unavailable (e.g. Neo4j 2.1) and when it's disabled. + # https://mix.fiftythree.com/aseemk/2471430 + # NOTE: This might need updating if the disabled 500 is fixed. + # https://github.com/neo4j/neo4j/issues/4138 + try + DB.checkPasswordChangeNeeded _ + + # If this worked, auth is available and enabled. + return + + catch err + # If we're right that this means auth is unavailable or disabled, + # we can verify that by querying the Neo4j version. + dbVersion = tryGetDbVersion _ + + if (err instanceof neo4j.ClientError) and + (err.message.match /^404 /) and (dbVersion < 2.2) + return disable 'Neo4j <2.2 detected' + + if (err instanceof neo4j.DatabaseError) and + (err.message.match /^500 /) and (dbVersion >= 2.2) + return disable 'Neo4j auth appears disabled' + + disable 'Error checking auth' + throw err + + it 'should fail when auth is required but not set' + + it 'should fail when auth is incorrect (username)' + + it 'should fail when auth is incorrect (password)' + + it 'should support checking whether a password change is needed', (_) -> + needed = DB.checkPasswordChangeNeeded _ + expect(needed).to.be.a 'boolean' + + it 'should support changing the current user’s password', (_) -> + DB.changePassword RANDOM_PASSWORD, _ + + it 'should reject empty and null new passwords', -> + cb = -> throw new Error 'Callback shouldn’t have been called!' + + for fn in [ + -> DB.changePassword null, cb + -> DB.changePassword '', cb + -> DB.changePassword {}, cb + -> DB.changePassword {password: null}, cb + -> DB.changePassword {password: ''}, cb + ] + expect(fn).to.throw TypeError, /Password required/ + + it 'should automatically update state on password changes', (_) -> + expect(DB.auth.password).to.equal RANDOM_PASSWORD + + # Verify with another password change needed check: + needed = DB.checkPasswordChangeNeeded _ + expect(needed).to.equal false + + it '(change password back)', (_) -> + DB.changePassword ORIGINAL_PASSWORD, _ + expect(DB.auth.password).to.equal ORIGINAL_PASSWORD diff --git a/test-new/fixtures/index._coffee b/test-new/fixtures/index._coffee index 5057ed0..691526d 100644 --- a/test-new/fixtures/index._coffee +++ b/test-new/fixtures/index._coffee @@ -7,8 +7,11 @@ $ = require 'underscore' {expect} = require 'chai' neo4j = require '../../' -@DB = - new neo4j.GraphDatabase process.env.NEO4J_URL or 'http://localhost:7474' +@DB = new neo4j.GraphDatabase + # Support specifying database info via environment variables, + # but assume Neo4j installation defaults. + url: process.env.NEO4J_URL or 'http://neo4j:neo4j@localhost:7474' + auth: process.env.NEO4J_AUTH # We fill these in, and cache them, the first time tests request them: @DB_VERSION_NUM = null From ddceaa2a0145c07568f1182edb4eb5384a8ebd50 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sat, 7 Mar 2015 23:49:13 -0500 Subject: [PATCH 048/121] v2 / API: fix semicolons in JS code snippets. --- API_v2.md | 55 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/API_v2.md b/API_v2.md index 2015070..09467fc 100644 --- a/API_v2.md +++ b/API_v2.md @@ -95,11 +95,12 @@ property on the `GraphDatabase` instance, so that subsequent requests will work. This will allow callers to make any API requests; no one will be blocked by this driver not supporting a particular API. -It'll also allow callers to interface with arbitrary plugins, -including custom ones. +It'll also allow callers to interface with arbitrary Neo4j plugins +(e.g. [neo4j-spatial](http://neo4j-contrib.github.io/spatial/#spatial-server-plugin)), +even custom ones. ```js -function cb(err, body) {}; +function cb(err, body) {} var req = db.http({method, path, headers, body, raw}, cb); ``` @@ -171,7 +172,7 @@ from database responses. **Let me make simple, parametrized Cypher queries.** ```js -function cb(err, results) {}; +function cb(err, results) {} var stream = db.cypher({query, params, headers, lean}, cb); @@ -254,8 +255,8 @@ class Transaction {_id, expiresAt, expiresIn, state} ``` ```js -function cbResults(err, results) {}; -function cbDone(err) {}; +function cbResults(err, results) {} +function cbDone(err) {} var stream = tx.cypher({query, params, headers, lean, commit}, cbResults); @@ -381,7 +382,7 @@ for those boilerplate Cypher queries. ### Labels ```js -function cb(err, labels) {}; +function cb(err, labels) {} db.getLabels(cb); ``` @@ -397,10 +398,10 @@ Labels are simple strings. ### Indexes ```js -function cbOne(err, index) {}; -function cbMany(err, indexes) {}; -function cbBool(err, bool) {}; -function cbDone(err) {}; +function cbOne(err, index) {} +function cbMany(err, indexes) {} +function cbBool(err, bool) {} +function cbDone(err) {} db.getIndexes(cbMany); // across all labels db.getIndexes({label}, cbMany); // for a particular label @@ -427,10 +428,10 @@ The design aims to be generic in order to support future constraint types, but it's still possible that the API may have to break when that happens. ```js -function cbOne(err, constraint) {}; -function cbMany(err, constraints) {}; -function cbBool(err, bool) {}; -function cbDone(err) {}; +function cbOne(err, constraint) {} +function cbMany(err, constraints) {} +function cbBool(err, bool) {} +function cbDone(err) {} db.getConstraints(cbMany); // across all labels db.getConstraints({label}, cbMany); // for a particular label @@ -452,8 +453,8 @@ Should multiple properties be supported? ### Misc ```js -function cbKeys(err, keys) {}; -function cbTypes(err, types) {}; +function cbKeys(err, keys) {} +function cbTypes(err, types) {} db.getPropertyKeys(cbKeys); db.getRelationshipTypes(cbTypes); @@ -470,9 +471,9 @@ This driver thus provides legacy indexing APIs. ### Management ```js -function cbOne(err, index) {}; -function cbMany(err, indexes) {}; -function cbDone(err) {}; +function cbOne(err, index) {} +function cbMany(err, indexes) {} +function cbDone(err) {} db.getLegacyNodeIndexes(cbMany); db.getLegacyNodeIndex({name}, cbOne); @@ -498,9 +499,9 @@ The `config` property is e.g. `{provider: 'lucene', type: 'fulltext'}`; ### Simple Usage ```js -function cbOne(err, node_or_rel) {}; -function cbMany(err, nodes_or_rels) {}; -function cbDone(err) {}; +function cbOne(err, node_or_rel) {} +function cbMany(err, nodes_or_rels) {} +function cbDone(err) {} db.addNodeToLegacyIndex({name, key, value, _id}, cbOne); db.getNodesFromLegacyIndex({name, key, value}, cbMany); // key-value lookup @@ -531,7 +532,7 @@ For adding existing nodes or relationships, simply pass `unique: true` to the `add` method. ```js -function cb(err, node_or_rel) {}; +function cb(err, node_or_rel) {} db.addNodeToLegacyIndex({name, key, value, _id, unique: true}, cb); db.addRelationshipToLegacyIndex({name, key, value, _id, unique: true}, cb); @@ -545,7 +546,7 @@ For creating new nodes or relationships, the `create` method below corresponds with "create or fail", while `getOrCreate` corresponds with "get or create": ```js -function cb(err, node_or_rel) {}; +function cb(err, node_or_rel) {} db.createNodeFromLegacyIndex({name, key, value, properties}, cb); db.getOrCreateNodeFromLegacyIndex({name, key, value, properties}, cb); @@ -567,8 +568,8 @@ just replace `LegacyIndex` with `LegacyAutoIndex` in all method names, then omit the `name` parameter. ```js -function cbOne(err, node_or_rel) {}; -function cbMany(err, nodes_or_rels) {}; +function cbOne(err, node_or_rel) {} +function cbMany(err, nodes_or_rels) {} db.getNodesFromLegacyAutoIndex({key, value}, cbMany); // key-value lookup db.getNodesFromLegacyAutoIndex({query}, cbMany); // arbitrary Lucene query From 1ad1745db2a677645606582bd337eb115619e5d7 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Mar 2015 15:11:01 -0400 Subject: [PATCH 049/121] v2 / Errors: add support for new error format outside of txs too, e.g. auth. --- lib-new/GraphDatabase.coffee | 2 +- lib-new/errors.coffee | 13 ++++++-- test-new/_auth._coffee | 61 +++++++++++++++++++++++++++++++++--- 3 files changed, 67 insertions(+), 9 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index f0c1d75..bf2934f 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -310,7 +310,7 @@ module.exports = class GraphDatabase # TODO: Is it possible to get back more than one error? # If so, is it fine for us to just use the first one? [error] = errors - err = Error._fromTransaction error + err = Error._fromObject error cb err, results diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 38ed513..ad4ccb1 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -17,6 +17,13 @@ class @Error extends Error return null if statusCode < 400 + # If this is a Neo4j v2-style error response, prefer that: + if body?.errors?.length + # TODO: Is it possible to get back more than one error? + # If so, is it fine for us to just use the first one? + [error] = body.errors + return @_fromObject error + # TODO: Do some status codes (or perhaps inner `exception` names) # signify Transient errors rather than Database ones? ErrorType = if statusCode >= 500 then 'Database' else 'Client' @@ -39,10 +46,10 @@ class @Error extends Error new ErrorClass message, body # - # Accepts the given error object from a transactional Cypher response, and - # creates and returns the appropriate Error instance for it. + # Accepts the given (Neo4j v2) error object, and creates and returns the + # appropriate Error instance for it. # - @_fromTransaction: (obj) -> + @_fromObject: (obj) -> # http://neo4j.com/docs/stable/rest-api-transactional.html#rest-api-handling-errors # http://neo4j.com/docs/stable/status-codes.html {code, message, stackTrace} = obj diff --git a/test-new/_auth._coffee b/test-new/_auth._coffee index c8cc931..6e9fa84 100644 --- a/test-new/_auth._coffee +++ b/test-new/_auth._coffee @@ -11,12 +11,18 @@ # If auth is disabled or not present, every test here will be skipped. # +$ = require 'underscore' crypto = require 'crypto' {expect} = require 'chai' fixtures = require './fixtures' neo4j = require '../' +## CONSTANTS + +AUTH_ERROR_CODE = 'Neo.ClientError.Security.AuthorizationFailed' + + ## SHARED STATE {DB} = fixtures @@ -47,6 +53,18 @@ tryGetDbVersion = (_) -> catch err NaN # since a boolean comparison of NaN with any number is false +# TODO: De-duplicate this kind of helper between our test suites: +expectAuthError = (err, message) -> + expect(err).to.be.an.instanceof neo4j.ClientError + expect(err.name).to.equal 'neo4j.ClientError' + expect(err.message).to.equal "[#{AUTH_ERROR_CODE}] #{message}" + + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j).to.eql {code: AUTH_ERROR_CODE, message} + ## TESTS @@ -85,11 +103,44 @@ describe 'Auth', -> disable 'Error checking auth' throw err - it 'should fail when auth is required but not set' - - it 'should fail when auth is incorrect (username)' - - it 'should fail when auth is incorrect (password)' + it 'should fail when auth is required but not set', (done) -> + db = new neo4j.GraphDatabase + url: DB.url + auth: {} # Explicitly clears auth + + # NOTE: Explicitly not using `db.checkPasswordChangeNeeded` since that + # rejects calls when no auth is set. + db.http '/db/data/', (err, data) -> + expect(err).to.exist() + expectAuthError err, 'No authorization header supplied.' + expect(data).to.not.exist() + done() + + it 'should fail when auth is incorrect (username)', (done) -> + db = new neo4j.GraphDatabase + url: DB.url + auth: $(DB.auth).clone() + + db.auth.username = RANDOM_PASSWORD + + db.checkPasswordChangeNeeded (err, bool) -> + expect(err).to.exist() + expectAuthError err, 'Invalid username or password.' + expect(bool).to.not.exist() + done() + + it 'should fail when auth is incorrect (password)', (done) -> + db = new neo4j.GraphDatabase + url: DB.url + auth: $(DB.auth).clone() + + db.auth.password = RANDOM_PASSWORD + + db.checkPasswordChangeNeeded (err, bool) -> + expect(err).to.exist() + expectAuthError err, 'Invalid username or password.' + expect(bool).to.not.exist() + done() it 'should support checking whether a password change is needed', (_) -> needed = DB.checkPasswordChangeNeeded _ From 48f30e8a2baaf489899a4abfa4b5f85db762e5b7 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 8 Mar 2015 15:44:26 -0400 Subject: [PATCH 050/121] v2 / Tests: simplify manual error callback assertions. --- test-new/cypher._coffee | 92 ++++++++++++++------------------ test-new/http._coffee | 80 ++++++++++++---------------- test-new/transactions._coffee | 99 +++++++++++++---------------------- 3 files changed, 111 insertions(+), 160 deletions(-) diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index b5f26c2..b63f4af 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -110,41 +110,33 @@ describe 'GraphDatabase::cypher', -> it 'should properly parse and throw Neo4j errors', (done) -> DB.cypher 'RETURN {foo}', (err, results) -> - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - - # Whether `results` are returned or not depends on the error; - # Neo4j will return an array if the query could be executed, - # and then it'll return whatever results it could manage to get - # before the error. In this case, the query began execution, - # so we expect an array, but no actual results. - expect(results).to.be.an 'array' - expect(results).to.be.empty() - - catch assertionErr - return done assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' + + # Whether `results` are returned or not depends on the error; + # Neo4j will return an array if the query could be executed, + # and then it'll return whatever results it could manage to get + # before the error. In this case, the query began execution, + # so we expect an array, but no actual results. + expect(results).to.be.an 'array' + expect(results).to.be.empty() done() it 'should properly return null result on syntax errors', (done) -> DB.cypher '(syntax error)', (err, results) -> - try - expect(err).to.exist() + expect(err).to.exist() - # Simplified error checking, since the message is complex: - expect(err).to.be.an.instanceOf neo4j.ClientError - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.code).to.equal \ - 'Neo.ClientError.Statement.InvalidSyntax' + # Simplified error checking, since the message is complex: + expect(err).to.be.an.instanceOf neo4j.ClientError + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.code).to.equal \ + 'Neo.ClientError.Statement.InvalidSyntax' - # Unlike the previous test case, since Neo4j could not be - # executed, no results should have been returned at all: - expect(results).to.not.exist() - - catch assertionErr - return done assertionErr + # Unlike the previous test case, since Neo4j could not be + # executed, no results should have been returned at all: + expect(results).to.not.exist() done() @@ -278,30 +270,26 @@ describe 'GraphDatabase::cypher', -> idR: TEST_REL._id ] , (err, results) -> - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', 'ParameterMissing', - 'Expected a parameter named foo' - - # NOTE: With batching, we *do* return any results that we - # received before the error, in case of an open transaction. - # This means that we'll always return an array here, and it'll - # have just as many elements as queries that returned an array - # before the error. In this case, a ParameterMissing error in - # the second query means the second array *was* returned (since - # Neo4j could begin executing the query; see note in the first - # error handling test case in this suite), so two results. - expect(results).to.be.an 'array' - expect(results).to.have.length 2 - - [resultsA, resultsB] = results - expect(resultsA).to.eql [ - a: TEST_NODE_A.properties - ] - expect(resultsB).to.be.empty() - - catch assertionErr - return done assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', 'ParameterMissing', + 'Expected a parameter named foo' + + # NOTE: With batching, we *do* return any results that we + # received before the error, in case of an open transaction. + # This means that we'll always return an array here, and it'll + # have just as many elements as queries that returned an array + # before the error. In this case, a ParameterMissing error in + # the second query means the second array *was* returned (since + # Neo4j could begin executing the query; see note in the first + # error handling test case in this suite), so two results. + expect(results).to.be.an 'array' + expect(results).to.have.length 2 + + [resultsA, resultsB] = results + expect(resultsA).to.eql [ + a: TEST_NODE_A.properties + ] + expect(resultsB).to.be.empty() done() diff --git a/test-new/http._coffee b/test-new/http._coffee index ec0013f..f7af261 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -72,16 +72,12 @@ describe 'GraphDatabase::http', -> method: 'POST' path: '/' , (err, body) -> - try - expect(err).to.exist() - expect(body).to.not.exist() + expect(err).to.exist() + expect(body).to.not.exist() - expectError err, neo4j.ClientError, - '405 Method Not Allowed response for POST /' - expect(err.neo4j).to.be.empty() - - catch assertionErr - return done assertionErr + expectError err, neo4j.ClientError, + '405 Method Not Allowed response for POST /' + expect(err.neo4j).to.be.empty() done() @@ -90,28 +86,24 @@ describe 'GraphDatabase::http', -> method: 'GET' path: '/db/data/node/-1' , (err, body) -> - try - expect(err).to.exist() - expect(body).to.not.exist() - - expectError err, neo4j.ClientError, '404 [NodeNotFoundException] - Cannot find node with id [-1] in database.' + expect(err).to.exist() + expect(body).to.not.exist() - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.exception).to.equal 'NodeNotFoundException' - expect(err.neo4j.fullname).to.equal ' - org.neo4j.server.rest.web.NodeNotFoundException' - expect(err.neo4j.message).to.equal ' - Cannot find node with id [-1] in database.' + expectError err, neo4j.ClientError, '404 [NodeNotFoundException] + Cannot find node with id [-1] in database.' - expect(err.neo4j.stacktrace).to.be.an 'array' - expect(err.neo4j.stacktrace).to.not.be.empty() - for line in err.neo4j.stacktrace - expect(line).to.be.a 'string' - expect(line).to.not.be.empty() + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.exception).to.equal 'NodeNotFoundException' + expect(err.neo4j.fullname).to.equal ' + org.neo4j.server.rest.web.NodeNotFoundException' + expect(err.neo4j.message).to.equal ' + Cannot find node with id [-1] in database.' - catch assertionErr - return done assertionErr + expect(err.neo4j.stacktrace).to.be.an 'array' + expect(err.neo4j.stacktrace).to.not.be.empty() + for line in err.neo4j.stacktrace + expect(line).to.be.a 'string' + expect(line).to.not.be.empty() done() @@ -141,24 +133,20 @@ describe 'GraphDatabase::http', -> path: '/' raw: true , (err, resp) -> - try - expect(err).to.exist() - expect(resp).to.not.exist() - - # NOTE: *Not* using `expectError` here, because we explicitly - # don't wrap native (non-Neo4j) errors. - expect(err).to.be.an.instanceOf Error - expect(err.name).to.equal 'Error' - expect(err.code).to.equal 'ENOTFOUND' - expect(err.syscall).to.equal 'getaddrinfo' - expect(err.message).to.contain "#{err.syscall} #{err.code}" - # NOTE: Node 0.12 adds the hostname to the message. - expect(err.stack).to.contain '\n' - expect(err.stack.split('\n')[0]).to.equal \ - "#{err.name}: #{err.message}" - - catch assertionErr - return done assertionErr + expect(err).to.exist() + expect(resp).to.not.exist() + + # NOTE: *Not* using `expectError` here, because we explicitly + # don't wrap native (non-Neo4j) errors. + expect(err).to.be.an.instanceOf Error + expect(err.name).to.equal 'Error' + expect(err.code).to.equal 'ENOTFOUND' + expect(err.syscall).to.equal 'getaddrinfo' + expect(err.message).to.contain "#{err.syscall} #{err.code}" + # NOTE: Node 0.12 adds the hostname to the message. + expect(err.stack).to.contain '\n' + expect(err.stack.split('\n')[0]).to.equal \ + "#{err.name}: #{err.message}" done() diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index ae53d45..96014f9 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -78,11 +78,8 @@ describe 'Transactions', -> expect(tx.state).to.equal 'pending' cb = (err, results) -> - try - expect(err).to.not.exist() - expect(tx.state).to.equal 'open' - catch assertionErr - return done assertionErr + expect(err).to.not.exist() + expect(tx.state).to.equal 'open' done() fn() @@ -364,12 +361,9 @@ describe 'Transactions', -> params: idA: TEST_NODE_A._id , (err, results) => - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'open' @@ -431,12 +425,9 @@ describe 'Transactions', -> idA: TEST_NODE_A._id commit: true , (err, results) => - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'rolled back' @@ -485,13 +476,10 @@ describe 'Transactions', -> params: props: {foo: null} , (err, results) => - try - expect(err).to.exist() - expectError err, - 'DatabaseError', 'Statement', 'ExecutionFailure', - 'scala.MatchError: (foo,null) (of class scala.Tuple2)' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, + 'DatabaseError', 'Statement', 'ExecutionFailure', + 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() expect(tx.state).to.equal 'rolled back' @@ -520,12 +508,9 @@ describe 'Transactions', -> # For precision, implementing this step without Streamline. do (cont=_) => tx.cypher 'RETURN {foo}', (err, results) => - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'open' @@ -541,12 +526,9 @@ describe 'Transactions', -> query: 'RETURN {foo}' commit: true , (err, results) => - try - expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'rolled back' @@ -565,13 +547,10 @@ describe 'Transactions', -> params: props: {foo: null} , (err, results) => - try - expect(err).to.exist() - expectError err, - 'DatabaseError', 'Statement', 'ExecutionFailure', - 'scala.MatchError: (foo,null) (of class scala.Tuple2)' - catch assertionErr - return cont assertionErr + expect(err).to.exist() + expectError err, + 'DatabaseError', 'Statement', 'ExecutionFailure', + 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() expect(tx.state).to.equal 'rolled back' @@ -638,31 +617,27 @@ describe 'Transactions', -> idA: TEST_NODE_A._id ] , (err, results) => - try - expect(err).to.exist() + expect(err).to.exist() - # Simplified error checking, since the message is complex: - expect(err).to.be.an.instanceOf neo4j.ClientError - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.code).to.equal \ - 'Neo.ClientError.Statement.InvalidSyntax' + # Simplified error checking, since the message is complex: + expect(err).to.be.an.instanceOf neo4j.ClientError + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.code).to.equal \ + 'Neo.ClientError.Statement.InvalidSyntax' - expect(results).to.be.an 'array' - expect(results).to.have.length 1 + expect(results).to.be.an 'array' + expect(results).to.have.length 1 - [result] = results + [result] = results - expect(result).to.be.an 'array' - expect(result).to.have.length 1 + expect(result).to.be.an 'array' + expect(result).to.have.length 1 - [{nodeA}] = result + [{nodeA}] = result - # We requested `lean: true`, so `nodeA` is just properties: - expect(nodeA.test).to.equal 'errors with batching' - expect(nodeA.i).to.equal 2 - - catch assertionErr - return cont assertionErr + # We requested `lean: true`, so `nodeA` is just properties: + expect(nodeA.test).to.equal 'errors with batching' + expect(nodeA.i).to.equal 2 cont() From da354ed58f34688e8dc3448d6ab246d25739baeb Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 9 Mar 2015 22:36:23 -0400 Subject: [PATCH 051/121] v2 / Tests: make our new-style `expectErrors` helper consistent. TODO: Move it to a shared util! --- test-new/cypher._coffee | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index b63f4af..47b1079 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -27,13 +27,34 @@ expectError = (err, classification, category, title, message) -> expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError expect(err.name).to.equal "neo4j.#{classification}" - expect(err.message).to.equal "[#{code}] #{message}" + # If the actual error message is multi-line, it includes the Neo4j stack + # trace; test that in a simple way by just checking the first line of the + # trace (subsequent lines can be different, e.g. "Caused by"), but also test + # that the first line of the message matches the expected message: + expect(err.message).to.be.a 'string' + [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' + expect(errMessageLine1).to.equal "[#{code}] #{message}" + expect(errMessageLine2).to.match /// + ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) + /// if errMessageLine2 + + expect(err.stack).to.be.a 'string' expect(err.stack).to.contain '\n' - expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" + expect(err.stack).to.contain "#{err.name}: #{err.message}" + [errStackLine1, ...] = err.stack.split '\n' + expect(errStackLine1).to.equal "#{err.name}: #{errMessageLine1}" expect(err.neo4j).to.be.an 'object' - expect(err.neo4j).to.eql {code, message} + expect(err.neo4j.code).to.equal code + + # If the actual error message was multi-line, that means it was the Neo4j + # stack trace, which can include a larger message than the returned one. + if errMessageLine2 + expect(err.neo4j.message).to.be.a 'string' + expect(message).to.contain err.neo4j.message + else + expect(err.neo4j.message).to.equal message ## TESTS From 517211369fadfc1414ee8f0deb83267ebb56f889 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 9 Mar 2015 11:52:23 -0400 Subject: [PATCH 052/121] v2 / Tests: work around Neo4j 2.2.0-RC01 bugs for now. --- test-new/cypher._coffee | 24 +++++++++++++++++---- test-new/http._coffee | 21 +++++++++++++++++-- test-new/transactions._coffee | 39 +++++++++++++++++++++++++++-------- 3 files changed, 69 insertions(+), 15 deletions(-) diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 47b1079..ae91a74 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -56,6 +56,24 @@ expectError = (err, classification, category, title, message) -> else expect(err.neo4j.message).to.equal message +# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as +# `DatabaseError` rather than `ClientError`. +# https://github.com/neo4j/neo4j/issues/4144 +expectParameterMissingError = (err) -> + try + expectError err, 'ClientError', 'Statement', 'ParameterMissing', + 'Expected a parameter named foo' + + catch assertionErr + # Check for the Neo4j 2.2.0-RC01 case, but if it's not, + # throw the original assertion error, not a new one. + try + expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', + 'org.neo4j.graphdb.QueryExecutionException: + Expected a parameter named foo' + catch doubleErr + throw assertionErr + ## TESTS @@ -132,8 +150,7 @@ describe 'GraphDatabase::cypher', -> it 'should properly parse and throw Neo4j errors', (done) -> DB.cypher 'RETURN {foo}', (err, results) -> expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' + expectParameterMissingError err # Whether `results` are returned or not depends on the error; # Neo4j will return an array if the query could be executed, @@ -292,8 +309,7 @@ describe 'GraphDatabase::cypher', -> ] , (err, results) -> expect(err).to.exist() - expectError err, 'ClientError', 'Statement', 'ParameterMissing', - 'Expected a parameter named foo' + expectParameterMissingError err # NOTE: With batching, we *do* return any results that we # received before the error, in case of an open transaction. diff --git a/test-new/http._coffee b/test-new/http._coffee index f7af261..14c35c0 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -89,8 +89,25 @@ describe 'GraphDatabase::http', -> expect(err).to.exist() expect(body).to.not.exist() - expectError err, neo4j.ClientError, '404 [NodeNotFoundException] - Cannot find node with id [-1] in database.' + # TEMP: Neo4j 2.2 responds here with a new-style error object, + # but it's currently a `DatabaseError` in 2.2.0-RC01. + # https://github.com/neo4j/neo4j/issues/4145 + try + expectError err, neo4j.ClientError, '404 [NodeNotFoundException] + Cannot find node with id [-1] in database.' + catch assertionErr + # Check for the Neo4j 2.2 case, but if this fails, + # throw the original assertion error, not this one. + try + expectError err, neo4j.DatabaseError, + '[Neo.DatabaseError.General.UnknownFailure] + Cannot find node with id [-1] in database.' + catch doubleErr + throw assertionErr + + # HACK: If the check succeeded, skip all the other assertions + # below for now; we'll need to consider rewriting this: + return done() expect(err.neo4j).to.be.an 'object' expect(err.neo4j.exception).to.equal 'NodeNotFoundException' diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 96014f9..0c6ef61 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -57,6 +57,27 @@ expectError = (err, classification, category, title, message) -> else expect(err.neo4j.message).to.equal message +# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as +# `DatabaseError` rather than `ClientError`. +# https://github.com/neo4j/neo4j/issues/4144 +# Returns whether we did have to account for this bug or not. +expectParameterMissingError = (err) -> + try + expectError err, 'ClientError', 'Statement', 'ParameterMissing', + 'Expected a parameter named foo' + return false + + catch assertionErr + # Check for the Neo4j 2.2.0-RC01 case, but if it's not, + # throw the original assertion error, not a new one. + try + expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', + 'org.neo4j.graphdb.QueryExecutionException: + Expected a parameter named foo' + return true + + throw assertionErr + ## TESTS @@ -362,8 +383,7 @@ describe 'Transactions', -> idA: TEST_NODE_A._id , (err, results) => expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' + expectParameterMissingError err cont() expect(tx.state).to.equal 'open' @@ -426,8 +446,7 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' + expectParameterMissingError err cont() expect(tx.state).to.equal 'rolled back' @@ -506,14 +525,17 @@ describe 'Transactions', -> expect(tx.state).to.equal 'open' # For precision, implementing this step without Streamline. + dbErred = null do (cont=_) => tx.cypher 'RETURN {foo}', (err, results) => expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' + dbErred = expectParameterMissingError err cont() - expect(tx.state).to.equal 'open' + # TEMP: Because Neo4j 2.2.0-RC01 incorrectly throws a `DatabaseError` + # for the above error (see `expectParameterMissingError` for details), + # the transaction does get rolled back. + expect(tx.state).to.equal (if dbErred then 'rolled back' else 'open') it 'should properly handle fatal client errors on an auto-commit first query', (_) -> @@ -527,8 +549,7 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - expectError err, 'ClientError', 'Statement', - 'ParameterMissing', 'Expected a parameter named foo' + expectParameterMissingError err cont() expect(tx.state).to.equal 'rolled back' From 38972c7154c026417a2da48ee7658e97ceab4786 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Mon, 9 Mar 2015 22:47:10 -0400 Subject: [PATCH 053/121] v2 / Tests: create helpers file for e.g. `expectError`. --- test-new/cypher._coffee | 65 +------------------ test-new/http._coffee | 49 +++----------- test-new/transactions._coffee | 77 ++-------------------- test-new/util/helpers._coffee | 117 ++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+), 171 deletions(-) create mode 100644 test-new/util/helpers._coffee diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index ae91a74..717096a 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -5,6 +5,7 @@ {expect} = require 'chai' fixtures = require './fixtures' +helpers = require './util/helpers' neo4j = require '../' @@ -15,66 +16,6 @@ neo4j = require '../' [TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] -## HELPERS - -# -# Asserts that the given object is an instance of the proper Neo4j Error -# subclass, representing the given transactional Neo4j error info. -# TODO: Consider consolidating with a similar helper in the `http` test suite. -# -expectError = (err, classification, category, title, message) -> - code = "Neo.#{classification}.#{category}.#{title}" - - expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError - expect(err.name).to.equal "neo4j.#{classification}" - - # If the actual error message is multi-line, it includes the Neo4j stack - # trace; test that in a simple way by just checking the first line of the - # trace (subsequent lines can be different, e.g. "Caused by"), but also test - # that the first line of the message matches the expected message: - expect(err.message).to.be.a 'string' - [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' - expect(errMessageLine1).to.equal "[#{code}] #{message}" - expect(errMessageLine2).to.match /// - ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) - /// if errMessageLine2 - - expect(err.stack).to.be.a 'string' - expect(err.stack).to.contain '\n' - expect(err.stack).to.contain "#{err.name}: #{err.message}" - [errStackLine1, ...] = err.stack.split '\n' - expect(errStackLine1).to.equal "#{err.name}: #{errMessageLine1}" - - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.code).to.equal code - - # If the actual error message was multi-line, that means it was the Neo4j - # stack trace, which can include a larger message than the returned one. - if errMessageLine2 - expect(err.neo4j.message).to.be.a 'string' - expect(message).to.contain err.neo4j.message - else - expect(err.neo4j.message).to.equal message - -# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as -# `DatabaseError` rather than `ClientError`. -# https://github.com/neo4j/neo4j/issues/4144 -expectParameterMissingError = (err) -> - try - expectError err, 'ClientError', 'Statement', 'ParameterMissing', - 'Expected a parameter named foo' - - catch assertionErr - # Check for the Neo4j 2.2.0-RC01 case, but if it's not, - # throw the original assertion error, not a new one. - try - expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', - 'org.neo4j.graphdb.QueryExecutionException: - Expected a parameter named foo' - catch doubleErr - throw assertionErr - - ## TESTS describe 'GraphDatabase::cypher', -> @@ -150,7 +91,7 @@ describe 'GraphDatabase::cypher', -> it 'should properly parse and throw Neo4j errors', (done) -> DB.cypher 'RETURN {foo}', (err, results) -> expect(err).to.exist() - expectParameterMissingError err + helpers.expectParameterMissingError err # Whether `results` are returned or not depends on the error; # Neo4j will return an array if the query could be executed, @@ -309,7 +250,7 @@ describe 'GraphDatabase::cypher', -> ] , (err, results) -> expect(err).to.exist() - expectParameterMissingError err + helpers.expectParameterMissingError err # NOTE: With batching, we *do* return any results that we # received before the error, in case of an open transaction. diff --git a/test-new/http._coffee b/test-new/http._coffee index 14c35c0..f0620d0 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -6,6 +6,7 @@ {expect} = require 'chai' fixtures = require './fixtures' fs = require 'fs' +helpers = require './util/helpers' http = require 'http' neo4j = require '../' @@ -35,19 +36,6 @@ expectNeo4jRoot = (body) -> expect(body).to.be.an 'object' expect(body).to.have.keys 'data', 'management' -# -# Asserts that the given object is a proper instance of the given Neo4j Error -# subclass, including with the given message. -# Additional checks, e.g. of the `neo4j` property's contents, are up to you. -# -expectError = (err, ErrorClass, message) -> - expect(err).to.be.an.instanceOf ErrorClass - expect(err.name).to.equal "neo4j.#{ErrorClass.name}" - expect(err.neo4j).to.be.an 'object' - expect(err.message).to.equal message - expect(err.stack).to.contain '\n' - expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" - ## TESTS @@ -75,9 +63,8 @@ describe 'GraphDatabase::http', -> expect(err).to.exist() expect(body).to.not.exist() - expectError err, neo4j.ClientError, + helpers.expectRawError err, 'ClientError', '405 Method Not Allowed response for POST /' - expect(err.neo4j).to.be.empty() done() @@ -93,35 +80,19 @@ describe 'GraphDatabase::http', -> # but it's currently a `DatabaseError` in 2.2.0-RC01. # https://github.com/neo4j/neo4j/issues/4145 try - expectError err, neo4j.ClientError, '404 [NodeNotFoundException] - Cannot find node with id [-1] in database.' + helpers.expectOldError err, 404, 'NodeNotFoundException', + 'org.neo4j.server.rest.web.NodeNotFoundException', + 'Cannot find node with id [-1] in database.' catch assertionErr # Check for the Neo4j 2.2 case, but if this fails, # throw the original assertion error, not this one. try - expectError err, neo4j.DatabaseError, - '[Neo.DatabaseError.General.UnknownFailure] - Cannot find node with id [-1] in database.' + helpers.expectError err, + 'DatabaseError', 'General', 'UnknownFailure', + 'Cannot find node with id [-1] in database.' catch doubleErr throw assertionErr - # HACK: If the check succeeded, skip all the other assertions - # below for now; we'll need to consider rewriting this: - return done() - - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.exception).to.equal 'NodeNotFoundException' - expect(err.neo4j.fullname).to.equal ' - org.neo4j.server.rest.web.NodeNotFoundException' - expect(err.neo4j.message).to.equal ' - Cannot find node with id [-1] in database.' - - expect(err.neo4j.stacktrace).to.be.an 'array' - expect(err.neo4j.stacktrace).to.not.be.empty() - for line in err.neo4j.stacktrace - expect(line).to.be.a 'string' - expect(line).to.not.be.empty() - done() it 'should support returning raw responses', (_) -> @@ -153,8 +124,8 @@ describe 'GraphDatabase::http', -> expect(err).to.exist() expect(resp).to.not.exist() - # NOTE: *Not* using `expectError` here, because we explicitly - # don't wrap native (non-Neo4j) errors. + # NOTE: *Not* using our `expectError` helpers here, because we + # explicitly don't wrap native (non-Neo4j) errors. expect(err).to.be.an.instanceOf Error expect(err.name).to.equal 'Error' expect(err.code).to.equal 'ENOTFOUND' diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 0c6ef61..9cdd49f 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -5,6 +5,7 @@ {expect} = require 'chai' fixtures = require './fixtures' +helpers = require './util/helpers' neo4j = require '../' @@ -15,70 +16,6 @@ neo4j = require '../' [TEST_NODE_A, TEST_NODE_B, TEST_REL] = [] - -## HELPERS - -# -# Asserts that the given object is an instance of the proper Neo4j Error -# subclass, representing the given transactional Neo4j error info. -# TODO: De-duplicate with same helper in Cypher test suite! -# -expectError = (err, classification, category, title, message) -> - code = "Neo.#{classification}.#{category}.#{title}" - - expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError - expect(err.name).to.equal "neo4j.#{classification}" - - # If the actual error message is multi-line, it includes the Neo4j stack - # trace; test that in a simple way by just checking the first line of the - # trace (subsequent lines can be different, e.g. "Caused by"), but also test - # that the first line of the message matches the expected message: - expect(err.message).to.be.a 'string' - [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' - expect(errMessageLine1).to.equal "[#{code}] #{message}" - expect(errMessageLine2).to.match /// - ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) - /// if errMessageLine2 - - expect(err.stack).to.be.a 'string' - expect(err.stack).to.contain '\n' - expect(err.stack).to.contain "#{err.name}: #{err.message}" - [errStackLine1, ...] = err.stack.split '\n' - expect(errStackLine1).to.equal "#{err.name}: #{errMessageLine1}" - - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j.code).to.equal code - - # If the actual error message was multi-line, that means it was the Neo4j - # stack trace, which can include a larger message than the returned one. - if errMessageLine2 - expect(err.neo4j.message).to.be.a 'string' - expect(message).to.contain err.neo4j.message - else - expect(err.neo4j.message).to.equal message - -# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as -# `DatabaseError` rather than `ClientError`. -# https://github.com/neo4j/neo4j/issues/4144 -# Returns whether we did have to account for this bug or not. -expectParameterMissingError = (err) -> - try - expectError err, 'ClientError', 'Statement', 'ParameterMissing', - 'Expected a parameter named foo' - return false - - catch assertionErr - # Check for the Neo4j 2.2.0-RC01 case, but if it's not, - # throw the original assertion error, not a new one. - try - expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', - 'org.neo4j.graphdb.QueryExecutionException: - Expected a parameter named foo' - return true - - throw assertionErr - - ## TESTS describe 'Transactions', -> @@ -383,7 +320,7 @@ describe 'Transactions', -> idA: TEST_NODE_A._id , (err, results) => expect(err).to.exist() - expectParameterMissingError err + helpers.expectParameterMissingError err cont() expect(tx.state).to.equal 'open' @@ -446,7 +383,7 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - expectParameterMissingError err + helpers.expectParameterMissingError err cont() expect(tx.state).to.equal 'rolled back' @@ -496,7 +433,7 @@ describe 'Transactions', -> props: {foo: null} , (err, results) => expect(err).to.exist() - expectError err, + helpers.expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() @@ -529,7 +466,7 @@ describe 'Transactions', -> do (cont=_) => tx.cypher 'RETURN {foo}', (err, results) => expect(err).to.exist() - dbErred = expectParameterMissingError err + dbErred = helpers.expectParameterMissingError err cont() # TEMP: Because Neo4j 2.2.0-RC01 incorrectly throws a `DatabaseError` @@ -549,7 +486,7 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - expectParameterMissingError err + helpers.expectParameterMissingError err cont() expect(tx.state).to.equal 'rolled back' @@ -569,7 +506,7 @@ describe 'Transactions', -> props: {foo: null} , (err, results) => expect(err).to.exist() - expectError err, + helpers.expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee new file mode 100644 index 0000000..58bea10 --- /dev/null +++ b/test-new/util/helpers._coffee @@ -0,0 +1,117 @@ +# +# NOTE: This file is within a `util` subdirectory, rather than within the +# top-level `test` directory, in order to not have Mocha treat it like a test. +# + +{expect} = require 'chai' +neo4j = require '../../' + + +# +# Helper used by all the below methods that covers all specific error cases. +# Asserts that the given error at least adheres to our base Error contract. +# +@_expectBaseError = (err, classification) -> + expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError + expect(err.name).to.equal "neo4j.#{classification}" + + expect(err.message).to.be.a 'string' + expect(err.stack).to.be.a 'string' + expect(err.stack).to.contain '\n' + + # NOTE: Chai doesn't have `beginsWith`, so approximating: + stackPrefix = "#{err.name}: #{err.message}" + expect(err.stack.slice 0, stackPrefix.length).to.equal stackPrefix + + +# +# Asserts that the given object is an instance of the appropriate Neo4j Error +# subclass, representing the given *new-style* Neo4j v2 error info. +# +@expectError = (err, classification, category, title, message) => + code = "Neo.#{classification}.#{category}.#{title}" + + @_expectBaseError err, classification + + # If the actual error message is multi-line, it includes the Neo4j stack + # trace; test that in a simple way by just checking the first line of the + # trace (subsequent lines can be different, e.g. "Caused by"), but also test + # that the first line of the message matches the expected message: + [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' + expect(errMessageLine1).to.equal "[#{code}] #{message}" + expect(errMessageLine2).to.match /// + ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) + /// if errMessageLine2 + + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j.code).to.equal code + + # If the actual error message was multi-line, that means it was the Neo4j + # stack trace, which can include a larger message than the returned one. + if errMessageLine2 + expect(err.neo4j.message).to.be.a 'string' + expect(message).to.contain err.neo4j.message + else + expect(err.neo4j.message).to.equal message + + +# +# Asserts that the given object is an instance of the appropriate Neo4j Error +# subclass, representing the given *old-style* Neo4j v1 error info. +# +# NOTE: This assumes this error is returned from an HTTP response. +# +@expectOldError = (err, statusCode, shortName, longName, message) -> + ErrorType = if statusCode >= 500 then 'Database' else 'Client' + @_expectBaseError err, "#{ErrorType}Error" + + expect(err.message).to.equal "#{statusCode} [#{shortName}] #{message}" + + expect(err.neo4j).to.be.an 'object' + expect(err.neo4j).to.contain + exception: shortName + fullname: longName + message: message + + expect(err.neo4j.stacktrace).to.be.an 'array' + expect(err.neo4j.stacktrace).to.not.be.empty() + for line in err.neo4j.stacktrace + expect(line).to.be.a 'string' + expect(line).to.not.be.empty() + + +# +# Asserts that the given object is an instance of the appropriate Neo4j Error +# subclass, with the given raw message. +# +# NOTE: This assumes no details info was returned by Neo4j. +# +@expectRawError = (err, classification, message) => + @_expectBaseError err, classification + expect(err.message).to.equal message + expect(err.neo4j).to.be.empty() # TODO: Should this really be the case? + + +# +# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as +# `DatabaseError` rather than `ClientError`: +# https://github.com/neo4j/neo4j/issues/4144 +# +# Returns whether we did have to account for this bug or not. +# +@expectParameterMissingError = (err) => + try + @expectError err, 'ClientError', 'Statement', 'ParameterMissing', + 'Expected a parameter named foo' + return false + + catch assertionErr + # Check for the Neo4j 2.2.0-RC01 case, but if it's not, + # throw the original assertion error, not a new one. + try + @expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', + 'org.neo4j.graphdb.QueryExecutionException: + Expected a parameter named foo' + return true + + throw assertionErr From 62c6d976c22ff01c27bb7e991b9c2cdec44c9a7e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Mar 2015 13:13:14 -0400 Subject: [PATCH 054/121] v2 / Tests: test io.js on Travis! --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 3850a5f..749ba0a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,7 @@ language: node_js node_js: + - "io.js" - "0.12" - "0.10" From 410e8ab2bdc73f56f165afa3eb76b37779d26a23 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Mar 2015 20:51:46 -0400 Subject: [PATCH 055/121] v2 / Tests: fix and improve HTTP streaming test. --- test-new/http._coffee | 46 ++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/test-new/http._coffee b/test-new/http._coffee index f0620d0..7317e1b 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -11,6 +11,11 @@ http = require 'http' neo4j = require '../' +## CONSTANTS + +FAKE_JSON_PATH = "#{__dirname}/fixtures/fake.json" + + ## SHARED STATE {DB} = fixtures @@ -140,13 +145,15 @@ describe 'GraphDatabase::http', -> it 'should support streaming', (done) -> opts = - method: 'PUT' - path: '/db/data/node/-1/properties' + method: 'POST' + path: '/db/data/node' headers: - # Node recommends setting this property when streaming: + # NOTE: It seems that Neo4j needs an explicit Content-Length, + # at least for requests to this `POST /db/data/node` endpoint. + 'Content-Length': (fs.statSync FAKE_JSON_PATH).size + # Ideally, we would instead send this header for streaming: # http://nodejs.org/api/http.html#http_request_write_chunk_encoding_callback - 'Transfer-Encoding': 'chunked' - 'X-Foo': 'Bar' # This one's just for testing/verifying + # 'Transfer-Encoding': 'chunked' req = DB.http opts @@ -162,8 +169,13 @@ describe 'GraphDatabase::http', -> req.on 'error', done # Now stream some fake JSON to the request: - fs.createReadStream "#{__dirname}/fixtures/fake.json" - .pipe req + # TODO: Why doesn't this work? + # fs.createReadStream(FAKE_JSON_PATH).pipe req + # TEMP: Instead, we have to manually pipe: + readStream = fs.createReadStream FAKE_JSON_PATH + readStream.on 'error', done + readStream.on 'data', (chunk) -> req.write chunk + readStream.on 'end', -> req.end() # Verify that the request fully waits for our stream to finish # before returning a response: @@ -173,7 +185,7 @@ describe 'GraphDatabase::http', -> # When the response is received, stream down its JSON too: req.on 'response', (resp) -> expect(finished).to.be.true() - expectResponse resp, 404 + expectResponse resp, 400 resp.setEncoding 'utf8' body = '' @@ -182,16 +194,14 @@ describe 'GraphDatabase::http', -> resp.on 'error', done resp.on 'close', -> done new Error 'Response closed!' resp.on 'end', -> - try - body = JSON.parse body - - # Simplified error parsing; just verifying stream: - expect(body).to.be.an 'object' - expect(body.exception).to.equal 'NodeNotFoundException' - expect(body.stacktrace).to.be.an 'array' - - catch err - return done err + body = JSON.parse body + + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'PropertyValueException' + expect(body.message).to.equal 'Could not set property "object", + unsupported type: {foo={bar=baz}}' + expect(body.stacktrace).to.be.an 'array' done() From f96188286cd6b6ecdb1d748f119a9159fb593064 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 10 Mar 2015 22:39:42 -0400 Subject: [PATCH 056/121] v2 / Tests: DRY up random string generation. --- test-new/_auth._coffee | 4 ++-- test-new/constructor._coffee | 5 +++-- test-new/fixtures/index._coffee | 9 ++------- test-new/util/helpers._coffee | 7 +++++++ 4 files changed, 14 insertions(+), 11 deletions(-) diff --git a/test-new/_auth._coffee b/test-new/_auth._coffee index 6e9fa84..251e029 100644 --- a/test-new/_auth._coffee +++ b/test-new/_auth._coffee @@ -12,9 +12,9 @@ # $ = require 'underscore' -crypto = require 'crypto' {expect} = require 'chai' fixtures = require './fixtures' +helpers = require './util/helpers' neo4j = require '../' @@ -28,7 +28,7 @@ AUTH_ERROR_CODE = 'Neo.ClientError.Security.AuthorizationFailed' {DB} = fixtures ORIGINAL_PASSWORD = DB.auth?.password -RANDOM_PASSWORD = crypto.randomBytes(16).toString('base64') +RANDOM_PASSWORD = helpers.getRandomStr() SUITE = null diff --git a/test-new/constructor._coffee b/test-new/constructor._coffee index b2ffa63..7c3cb5f 100644 --- a/test-new/constructor._coffee +++ b/test-new/constructor._coffee @@ -5,6 +5,7 @@ $ = require 'underscore' {expect} = require 'chai' {GraphDatabase} = require '../' +helpers = require './util/helpers' ## CONSTANTS @@ -117,8 +118,8 @@ describe 'GraphDatabase::constructor', -> it 'should prefer separate auth option over auth in the URL (and should clear auth in URL then)', -> host = 'auth.test:9876' - wrong1 = "#{Math.random()}"[2..] - wrong2 = "#{Math.random()}"[2..] + wrong1 = helpers.getRandomStr() + wrong2 = helpers.getRandomStr() db = new GraphDatabase url: "https://#{wrong1}:#{wrong2}@#{host}" diff --git a/test-new/fixtures/index._coffee b/test-new/fixtures/index._coffee index 691526d..9fb21e8 100644 --- a/test-new/fixtures/index._coffee +++ b/test-new/fixtures/index._coffee @@ -5,6 +5,7 @@ $ = require 'underscore' {expect} = require 'chai' +helpers = require '../util/helpers' neo4j = require '../../' @DB = new neo4j.GraphDatabase @@ -40,19 +41,13 @@ neo4j = require '../../' throw new Error "*** node-neo4j v2 supports Neo4j v2+ only, and you’re running Neo4j v1. These tests will fail! ***" -# -# Returns a random string. -# -@getRandomStr = -> - "#{Math.random()}"[2..] - # # Creates and returns a property bag (dictionary) with unique, random test data # for the given test suite (pass the suite's Node `module`). # @createTestProperties = (suite) => suite: suite.filename - rand: @getRandomStr() + rand: helpers.getRandomStr() # # Creates and returns a new Node instance with unique, random test data for the diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee index 58bea10..a37471d 100644 --- a/test-new/util/helpers._coffee +++ b/test-new/util/helpers._coffee @@ -115,3 +115,10 @@ neo4j = require '../../' return true throw assertionErr + + +# +# Returns a random string. +# +@getRandomStr = -> + "#{Math.random()}"[2..] From 96e3432f1ad5bc546add6d2360416e2e2e363c84 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 11 Mar 2015 00:37:18 -0400 Subject: [PATCH 057/121] v2 / Errors: work around another 2.2 regression; robustify test helper. --- lib-new/errors.coffee | 8 +++++ test-new/http._coffee | 3 +- test-new/util/helpers._coffee | 55 ++++++++++++++++++++++------------- 3 files changed, 45 insertions(+), 21 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index ad4ccb1..42e1efd 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -1,4 +1,5 @@ $ = require 'underscore' +assert = require 'assert' http = require 'http' class @Error extends Error @@ -50,6 +51,13 @@ class @Error extends Error # appropriate Error instance for it. # @_fromObject: (obj) -> + # NOTE: Neo4j seems to return both `stackTrace` and `stacktrace`. + # https://github.com/neo4j/neo4j/issues/4145#issuecomment-78203290 + # Normalizing to consistent `stackTrace` before we parse below! + if obj.stacktrace? and not obj.stackTrace? + obj.stackTrace = obj.stacktrace + delete obj.stacktrace + # http://neo4j.com/docs/stable/rest-api-transactional.html#rest-api-handling-errors # http://neo4j.com/docs/stable/status-codes.html {code, message, stackTrace} = obj diff --git a/test-new/http._coffee b/test-new/http._coffee index 7317e1b..b61d77d 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -94,7 +94,8 @@ describe 'GraphDatabase::http', -> try helpers.expectError err, 'DatabaseError', 'General', 'UnknownFailure', - 'Cannot find node with id [-1] in database.' + 'org.neo4j.server.rest.web.NodeNotFoundException: + Cannot find node with id [-1] in database.' catch doubleErr throw assertionErr diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee index a37471d..221e52f 100644 --- a/test-new/util/helpers._coffee +++ b/test-new/util/helpers._coffee @@ -7,11 +7,20 @@ neo4j = require '../../' +# +# Chai doesn't have a `beginsWith` assertion, so this approximates that: +# Asserts that the given string begins with the given prefix. +# +@expectPrefix = (str, prefix) -> + expect(str).to.be.a 'string' + expect(str.slice 0, prefix.length).to.equal prefix + + # # Helper used by all the below methods that covers all specific error cases. # Asserts that the given error at least adheres to our base Error contract. # -@_expectBaseError = (err, classification) -> +@_expectBaseError = (err, classification) => expect(err).to.be.an.instanceOf neo4j[classification] # e.g. DatabaseError expect(err.name).to.equal "neo4j.#{classification}" @@ -19,9 +28,7 @@ neo4j = require '../../' expect(err.stack).to.be.a 'string' expect(err.stack).to.contain '\n' - # NOTE: Chai doesn't have `beginsWith`, so approximating: - stackPrefix = "#{err.name}: #{err.message}" - expect(err.stack.slice 0, stackPrefix.length).to.equal stackPrefix + @expectPrefix err.stack, "#{err.name}: #{err.message}" # @@ -30,29 +37,37 @@ neo4j = require '../../' # @expectError = (err, classification, category, title, message) => code = "Neo.#{classification}.#{category}.#{title}" + codePlusMessage = "[#{code}] #{message}" @_expectBaseError err, classification - # If the actual error message is multi-line, it includes the Neo4j stack - # trace; test that in a simple way by just checking the first line of the - # trace (subsequent lines can be different, e.g. "Caused by"), but also test - # that the first line of the message matches the expected message: - [errMessageLine1, errMessageLine2, ...] = err.message.split '\n' - expect(errMessageLine1).to.equal "[#{code}] #{message}" - expect(errMessageLine2).to.match /// - ^ \s+ at\ [^(]+ \( [^)]+ [.](java|scala):\d+ \) - /// if errMessageLine2 - expect(err.neo4j).to.be.an 'object' expect(err.neo4j.code).to.equal code - # If the actual error message was multi-line, that means it was the Neo4j - # stack trace, which can include a larger message than the returned one. - if errMessageLine2 - expect(err.neo4j.message).to.be.a 'string' - expect(message).to.contain err.neo4j.message - else + # Neo4j can return its own Java stack trace, which changes our logic. + # So check for the simpler case first, then short-circuit: + if not err.neo4j.stackTrace + expect(err.message).to.equal codePlusMessage expect(err.neo4j.message).to.equal message + return + + # Otherwise, we construct our own stack trace from the Neo4j stack trace, + # by setting our message to the Neo4j stack trace. + # Neo4j stack traces can have messages with slightly more detail than the + # returned `message` property, so we test that via "contains". + expect(err.neo4j.stackTrace).to.be.a 'string' + expect(err.neo4j.message).to.be.a 'string' + expect(message).to.contain err.neo4j.message + + # Finally, we test that our returned message indeed includes the Neo4j stack + # trace, after the expected message part (which can be multi-line). + # We test just the first line of the stack trace for simplicity. + # (Subsequent lines can be different, e.g. "Caused by ..."). + @expectPrefix err.message, codePlusMessage + [errMessageStackLine1, ...] = + (err.message.slice 0, codePlusMessage.length).split '\n' + [neo4jStackTraceLine1, ...] = err.neo4j.stackTrace.split '\n' + expect(errMessageStackLine1).to.contain neo4jStackTraceLine1.trim() # From fd4f311c8dac1f50adf2ab8efd860ae1c82fee19 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 11 Mar 2015 01:24:11 -0400 Subject: [PATCH 058/121] v2 / Tests: add helper for HTTP errors. --- test-new/util/helpers._coffee | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee index 221e52f..934f4e4 100644 --- a/test-new/util/helpers._coffee +++ b/test-new/util/helpers._coffee @@ -4,6 +4,7 @@ # {expect} = require 'chai' +http = require 'http' neo4j = require '../../' @@ -97,16 +98,37 @@ neo4j = require '../../' # # Asserts that the given object is an instance of the appropriate Neo4j Error -# subclass, with the given raw message. +# subclass, with the given raw message (which can be a fuzzy regex too). # # NOTE: This assumes no details info was returned by Neo4j. # @expectRawError = (err, classification, message) => @_expectBaseError err, classification - expect(err.message).to.equal message + + if typeof message is 'string' + expect(err.message).to.equal message + else if message instanceof RegExp + expect(err.message).to.match message + else + throw new Error "Unrecognized type of expected `message`: + #{typeof message} / #{message?.constructor.name}" + expect(err.neo4j).to.be.empty() # TODO: Should this really be the case? +# +# Asserts that the given object is a simple HTTP error for the given response +# status code, e.g. 404 or 501. +# +@expectHttpError = (err, statusCode) => + ErrorType = if statusCode >= 500 then 'Database' else 'Client' + statusText = http.STATUS_CODES[statusCode] # E.g. "Not Found" + + @expectRawError err, "#{ErrorType}Error", /// + ^ #{statusCode}\ #{statusText}\ response\ for\ [A-Z]+\ /.* $ + /// + + # # TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as # `DatabaseError` rather than `ClientError`: From fd2b488b738437f3198d938ca2bc4e8e443489e2 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 11 Mar 2015 01:26:57 -0400 Subject: [PATCH 059/121] v2 / Indexes: implement and test! --- API_v2.md | 7 +- lib-new/GraphDatabase.coffee | 73 +++++++++- lib-new/Index.coffee | 25 ++++ lib-new/exports.coffee | 1 + test-new/indexes._coffee | 263 +++++++++++++++++++++++++++++++++++ test-new/schema._coffee | 5 +- 6 files changed, 369 insertions(+), 5 deletions(-) create mode 100644 lib-new/Index.coffee create mode 100644 test-new/indexes._coffee diff --git a/API_v2.md b/API_v2.md index 09467fc..2201fb2 100644 --- a/API_v2.md +++ b/API_v2.md @@ -407,7 +407,7 @@ db.getIndexes(cbMany); // across all labels db.getIndexes({label}, cbMany); // for a particular label db.hasIndex({label, property}, cbBool); db.createIndex({label, property}, cbOne); -db.deleteIndex({label, property}, cbDone); +db.dropIndex({label, property}, cbDone); ``` Returned indexes are minimal `Index` objects: @@ -420,6 +420,11 @@ TODO: Neo4j's REST API actually takes and returns *arrays* of properties, but AFAIK, all indexes today only deal with a single property. Should multiple properties be supported? +TODO: Today, there's no need for a `db.getIndex()` method, since the parameters +you need to fetch an index are the same ones that are returned. +But if Neo4j adds extra info to the response (e.g. online/offline status), +we should add that method, along with that extra info in our `Index` class. + ### Constraints The only constraint type implemented by Neo4j today is the uniqueness diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index bf2934f..ee4d7ac 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,6 +1,7 @@ $ = require 'underscore' assert = require 'assert' {Error} = require './errors' +Index = require './Index' lib = require '../package.json' Node = require './Node' Relationship = require './Relationship' @@ -350,7 +351,77 @@ module.exports = class GraphDatabase path: '/db/data/relationship/types' , cb - # TODO: Indexes + + ## INDEXES + + getIndexes: (opts={}, cb) -> + # Support passing no options at all, to mean "across all labels": + if typeof opts is 'function' + cb = opts + opts = {} + + # Also support passing a label directory: + if typeof opts is 'string' + opts = {label: opts} + + {label} = opts + + # Support both querying for a given label, and across all labels: + path = '/db/data/schema/index' + path += "/#{encodeURIComponent label}" if label + + @http + method: 'GET' + path: path + , (err, indexes) -> + cb err, indexes?.map Index._fromRaw + + hasIndex: (opts={}, cb) -> + {label, property} = opts + + if not (label and property) + throw new TypeError \ + 'Label and property required to query whether an index exists.' + + # NOTE: This is just a convenience method; there is no REST API endpoint + # for this directly (surprisingly, since there is for constraints). + @getIndexes {label}, (err, indexes) -> + if err + cb err + else + cb null, indexes.some (index) -> + index.label is label and index.property is property + + createIndex: (opts={}, cb) -> + {label, property} = opts + + if not (label and property) + throw new TypeError \ + 'Label and property required to create an index.' + + @http + method: 'POST' + path: "/db/data/schema/index/#{encodeURIComponent label}" + body: {'property_keys': [property]} + , (err, index) -> + cb err, if index then Index._fromRaw index + + dropIndex: (opts={}, cb) -> + {label, property} = opts + + if not (label and property) + throw new TypeError 'Label and property required to drop an index.' + + # This endpoint is void, i.e. returns nothing: + # http://neo4j.com/docs/stable/rest-api-schema-indexes.html#rest-api-drop-index + # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + @http + method: 'DELETE' + path: "/db/data/schema/index\ + /#{encodeURIComponent label}/#{encodeURIComponent property}" + , cb + + # TODO: Constraints # TODO: Legacy indexing diff --git a/lib-new/Index.coffee b/lib-new/Index.coffee new file mode 100644 index 0000000..bb89c0b --- /dev/null +++ b/lib-new/Index.coffee @@ -0,0 +1,25 @@ + +module.exports = class Index + + constructor: (opts={}) -> + {@label, @property} = opts + + equals: (other) -> + (other instanceof Index) and (@label is other.label) and + (@property is other.property) + + toString: -> + "INDEX ON :#{@label}(#{@property})" # E.g. "INDEX ON :User(email)" + + # + # Accepts the given raw JSON from Neo4j's REST API representing an index, + # and creates and returns a Index instance from it. + # + @_fromRaw: (obj) -> + {label, property_keys} = obj + + # TODO: Neo4j always returns an array of property keys, but only one + # property key is supported today. Do we need to support multiple? + [property] = property_keys + + return new Index {label, property} diff --git a/lib-new/exports.coffee b/lib-new/exports.coffee index 46e700f..fd223e5 100644 --- a/lib-new/exports.coffee +++ b/lib-new/exports.coffee @@ -5,5 +5,6 @@ $(exports).extend Node: require './Node' Relationship: require './Relationship' Transaction: require './Transaction' + Index: require './Index' $(exports).extend require './errors' diff --git a/test-new/indexes._coffee b/test-new/indexes._coffee new file mode 100644 index 0000000..b17a6c5 --- /dev/null +++ b/test-new/indexes._coffee @@ -0,0 +1,263 @@ +# +# Tests for indexing, e.g. creating, retrieivng, and dropping indexes. +# In the process, also tests that indexing actually has an effect. +# + +{expect} = require 'chai' +fixtures = require './fixtures' +helpers = require './util/helpers' +neo4j = require '../' + + +## SHARED STATE + +{DB, TEST_LABEL} = fixtures + +[TEST_NODE_A, TEST_REL, TEST_NODE_B] = [] + +# Important: we generate a random property to guarantee no index exists yet. +TEST_PROP = "index_#{helpers.getRandomStr()}" +TEST_VALUE = 'test' + +# These are the indexes that exist before this test suite runs... +ORIG_INDEXES_ALL = null # ...Across all labels +ORIG_INDEXES_LABEL = null # ...On our TEST_LABEL + +# And this is the index we create: +TEST_INDEX = null + +# Cypher query + params to test our index: +TEST_CYPHER = + query: """ + MATCH (n:#{TEST_LABEL}) + USING INDEX n:#{TEST_LABEL}(#{TEST_PROP}) + WHERE n.#{TEST_PROP} = {value} + RETURN n + ORDER BY ID(n) + """ + params: + value: TEST_VALUE + + +## HELPERS + +expectIndex = (index, label, property) -> + expect(index).to.be.an.instanceOf neo4j.Index + expect(index.label).to.equal label if label + expect(index.property).to.equal property if property + +expectIndexes = (indexes, label) -> + expect(indexes).to.be.an 'array' + for index in indexes + expectIndex index, label + + +## TESTS + +describe 'Indexes', -> + + # IMPORTANT: Mocha requires all test steps to be at the same nesting level + # for them to all execute in order. Hence "setup" and "teardown" wrappers. + + describe '(setup)', -> + + it '(create test nodes)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ + + it '(set test properties)', (_) -> + DB.cypher + query: """ + START n = node({ids}) + SET n.#{TEST_PROP} = {value} + """ + params: + ids: [TEST_NODE_A._id, TEST_NODE_B._id] + value: TEST_VALUE + , _ + + # Update our local instances too: + TEST_NODE_A.properties[TEST_PROP] = TEST_VALUE + TEST_NODE_B.properties[TEST_PROP] = TEST_VALUE + + + describe '(before index created)', -> + + it 'should support listing all indexes', (_) -> + indexes = DB.getIndexes _ + expectIndexes indexes + + ORIG_INDEXES_ALL = indexes + + it 'should support listing indexes for a particular label', (_) -> + indexes = DB.getIndexes TEST_LABEL, _ + expectIndexes indexes, TEST_LABEL + + ORIG_INDEXES_LABEL = indexes + + it 'should support querying for specific index', (_) -> + bool = DB.hasIndex + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal false + + it '(verify index doesn’t exist yet)', (done) -> + DB.cypher TEST_CYPHER, (err, results) -> + expect(err).to.exist() + expect(results).to.not.exist() + + helpers.expectError err, + 'ClientError', 'Schema', 'NoSuchIndex', """ + No such index found. + Label: `#{TEST_LABEL}` + Property name: `#{TEST_PROP}` + """ + + done() + + it 'should support creating index', (_) -> + index = DB.createIndex + label: TEST_LABEL + property: TEST_PROP + , _ + + expectIndex index, TEST_LABEL, TEST_PROP + + TEST_INDEX = index + + + describe '(after index created)', -> + + it '(verify by re-listing all indexes)', (_) -> + indexes = DB.getIndexes _ + expect(indexes).to.have.length ORIG_INDEXES_ALL.length + 1 + expect(indexes).to.contain TEST_INDEX + + it '(verify by re-listing indexes for test label)', (_) -> + indexes = DB.getIndexes TEST_LABEL, _ + expect(indexes).to.have.length ORIG_INDEXES_LABEL.length + 1 + expect(indexes).to.contain TEST_INDEX + + it '(verify by re-querying specific test index)', (_) -> + bool = DB.hasIndex + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal true + + # TODO: This sometimes fails because the index hasn't come online yet, + # but Neo4j's REST API doesn't return index online/offline status. + # We may need to change this to retry/poll for some time. + it.skip '(verify with test query)', (_) -> + results = DB.cypher TEST_CYPHER, _ + + expect(results).to.eql [ + n: TEST_NODE_A + , + n: TEST_NODE_B + ] + + it 'should throw on create of already-created index', (done) -> + DB.createIndex + label: TEST_LABEL + property: TEST_PROP + , (err, index) -> + expMessage = "There already exists an index + for label '#{TEST_LABEL}' on property '#{TEST_PROP}'." + + # Neo4j 2.2 returns a proper new-style error object for this + # case, but previous versions return an old-style error. + try + helpers.expectError err, 'ClientError', 'Schema', + 'IndexAlreadyExists', expMessage + catch assertionErr + # Check for the older case, but in case it fails, + # throw the original assertion error, not a new one. + try + helpers.expectOldError err, 409, + 'ConstraintViolationException', + 'org.neo4j.graphdb.ConstraintViolationException', + expMessage + catch doubleErr + throw assertionErr + + expect(index).to.not.exist() + done() + + it 'should support dropping index', (_) -> + DB.dropIndex + label: TEST_LABEL + property: TEST_PROP + , _ + + + describe '(after index dropped)', -> + + it '(verify by re-listing all indexes)', (_) -> + indexes = DB.getIndexes _ + expect(indexes).to.eql ORIG_INDEXES_ALL + + it '(verify by re-listing indexes for test label)', (_) -> + indexes = DB.getIndexes TEST_LABEL, _ + expect(indexes).to.eql ORIG_INDEXES_LABEL + + it '(verify by re-querying specific test index)', (_) -> + bool = DB.hasIndex + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal false + + it 'should throw on drop of already-dropped index', (done) -> + DB.dropIndex + label: TEST_LABEL + property: TEST_PROP + , (err) -> + helpers.expectHttpError err, 404 + done() + + + describe '(misc)', -> + + it 'should require both label and property to query specific index', -> + for fn in [ + -> DB.hasIndex null, -> + -> DB.hasIndex '', -> + -> DB.hasIndex {}, -> + -> DB.hasIndex TEST_LABEL, -> + -> DB.hasIndex {label: TEST_LABEL}, -> + -> DB.hasIndex {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + it 'should require both label and property to create index', -> + for fn in [ + -> DB.createIndex null, -> + -> DB.createIndex '', -> + -> DB.createIndex {}, -> + -> DB.createIndex TEST_LABEL, -> + -> DB.createIndex {label: TEST_LABEL}, -> + -> DB.createIndex {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + it 'should require both label and property to drop index', -> + for fn in [ + -> DB.dropIndex null, -> + -> DB.dropIndex '', -> + -> DB.dropIndex {}, -> + -> DB.dropIndex TEST_LABEL, -> + -> DB.dropIndex {label: TEST_LABEL}, -> + -> DB.dropIndex {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + + describe '(teardown)', -> + + it '(delete test nodes)', (_) -> + fixtures.deleteTestGraph module, _ diff --git a/test-new/schema._coffee b/test-new/schema._coffee index 68260a1..04a311f 100644 --- a/test-new/schema._coffee +++ b/test-new/schema._coffee @@ -1,7 +1,6 @@ # -# Tests for schema management, e.g. retrieving labels, property keys, and -# relationship types. Could also encompass constraints and (schema) indexes, -# but that could be its own test suite too if it goes deeper. +# Tests for basic schema management, e.g. retrieving labels, property keys, and +# relationship types. This does *not* cover indexes and constraints. # {expect} = require 'chai' From 54d2e9e474f1e690ac72d79a5ab873414bc9e14c Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 11 Mar 2015 23:55:18 -0400 Subject: [PATCH 060/121] v2 / Constraints: implement and test! --- API_v2.md | 15 +- lib-new/Constraint.coffee | 35 +++++ lib-new/GraphDatabase.coffee | 103 +++++++++++++- lib-new/exports.coffee | 1 + test-new/constraints._coffee | 260 +++++++++++++++++++++++++++++++++++ 5 files changed, 408 insertions(+), 6 deletions(-) create mode 100644 lib-new/Constraint.coffee create mode 100644 test-new/constraints._coffee diff --git a/API_v2.md b/API_v2.md index 2201fb2..14aa53f 100644 --- a/API_v2.md +++ b/API_v2.md @@ -428,9 +428,9 @@ we should add that method, along with that extra info in our `Index` class. ### Constraints The only constraint type implemented by Neo4j today is the uniqueness -constraint, so this API defaults to that. -The design aims to be generic in order to support future constraint types, -but it's still possible that the API may have to break when that happens. +constraint, so this API assumes that. +Because of that, it's possible that this API may have to break whenever new +constraint types are added (whatever they may be). ```js function cbOne(err, constraint) {} @@ -442,19 +442,24 @@ db.getConstraints(cbMany); // across all labels db.getConstraints({label}, cbMany); // for a particular label db.hasConstraint({label, property}, cbBool); db.createConstraint({label, property}, cbOne); -db.deleteConstraint({label, property}, cbDone); +db.dropConstraint({label, property}, cbDone); ``` Returned constraints are minimal `Constraint` objects: ```coffee -class Constraint {label, type, property} +class Constraint {label, property} ``` TODO: Neo4j's REST API actually takes and returns *arrays* of properties, but uniqueness constraints today only deal with a single property. Should multiple properties be supported? +TODO: Today, there's no need for a `db.getConstraint()` method, since the +parameters you need to fetch a constraint are the same ones that are returned. +But if Neo4j adds extra info to the response (e.g. online/offline status), +we should add that method, along with that extra info in our `Constraint` class. + ### Misc ```js diff --git a/lib-new/Constraint.coffee b/lib-new/Constraint.coffee new file mode 100644 index 0000000..a31dbe5 --- /dev/null +++ b/lib-new/Constraint.coffee @@ -0,0 +1,35 @@ +lib = require '../package.json' + + +module.exports = class Constraint + + constructor: (opts={}) -> + {@label, @property} = opts + + equals: (other) -> + (other instanceof Constraint) and (@label is other.label) and + (@property is other.property) + + toString: -> + # E.g. "CONSTRAINT ON (user:User) ASSERT user.email IS UNIQUE" + node = @label.toLowerCase() + "CONSTRAINT ON (#{node}:#{@label}) + ASSERT #{node}.#{@property} IS UNIQUE" + + # + # Accepts the given raw JSON from Neo4j's REST API representing a + # constraint, and creates and returns a Constraint instance from it. + # + @_fromRaw: (obj) -> + {type, label, property_keys} = obj + + if type isnt 'UNIQUENESS' + console.warn "Unrecognized constraint type encountered: #{type}. + node-neo4j v#{lib.version} doesn’t know how to handle these. + Continuing as if it’s a UNIQUENESS constraint..." + + # TODO: Neo4j always returns an array of property keys, but only one + # property key is supported today. Do we need to support multiple? + [property] = property_keys + + return new Constraint {label, property} diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index ee4d7ac..3ee2018 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -1,5 +1,6 @@ $ = require 'underscore' assert = require 'assert' +Constraint = require './Constraint' {Error} = require './errors' Index = require './Index' lib = require '../package.json' @@ -422,7 +423,107 @@ module.exports = class GraphDatabase , cb - # TODO: Constraints + ## CONSTRAINTS + + getConstraints: (opts={}, cb) -> + # Support passing no options at all, to mean "across all labels": + if typeof opts is 'function' + cb = opts + opts = {} + + # Also support passing a label directory: + if typeof opts is 'string' + opts = {label: opts} + + # TODO: We may need to support querying within a particular `type` too, + # if any other types beyond uniqueness get added. + {label} = opts + + # Support both querying for a given label, and across all labels. + # + # NOTE: We're explicitly *not* assuming uniqueness type here, since we + # couldn't achieve consistency with vs. without a label provided. + # (The `/uniqueness` part of the path can only come after a label.) + # + path = '/db/data/schema/constraint' + path += "/#{encodeURIComponent label}" if label + + @http + method: 'GET' + path: path + , (err, constraints) -> + cb err, constraints?.map Constraint._fromRaw + + hasConstraint: (opts={}, cb) -> + # TODO: We may need to support an additional `type` param too, + # if any other types beyond uniqueness get added. + {label, property} = opts + + if not (label and property) + throw new TypeError 'Label and property required to query + whether a constraint exists.' + + # NOTE: A REST API endpoint *does* exist to get a specific constraint: + # http://neo4j.com/docs/stable/rest-api-schema-constraints.html + # But it (a) returns an array, and (b) throws a 404 if no constraint. + # For those reasons, it's actually easier to just fetch all constraints; + # no error handling needed, and array processing either way. + # + # NOTE: We explicitly *are* assuming uniqueness type here, since we + # also would if we were querying for a specific constraint. + # (The `/uniqueness` part of the path comes before the property.) + # + @http + method: 'GET' + path: "/db/data/schema/constraint\ + /#{encodeURIComponent label}/uniqueness" + , (err, constraints) -> + if err + cb err + else + cb null, constraints.some (constraint) -> + constraint = Constraint._fromRaw constraint + constraint.label is label and + constraint.property is property + + createConstraint: (opts={}, cb) -> + # TODO: We may need to support an additional `type` param too, + # if any other types beyond uniqueness get added. + {label, property} = opts + + if not (label and property) + throw new TypeError \ + 'Label and property required to create a constraint.' + + # NOTE: We explicitly *are* assuming uniqueness type here, since + # that's our only option today for creating constraints. + @http + method: 'POST' + path: "/db/data/schema/constraint\ + /#{encodeURIComponent label}/uniqueness" + body: {'property_keys': [property]} + , (err, constraint) -> + cb err, if constraint then Constraint._fromRaw constraint + + dropConstraint: (opts={}, cb) -> + # TODO: We may need to support an additional `type` param too, + # if any other types beyond uniqueness get added. + {label, property} = opts + + if not (label and property) + throw new TypeError \ + 'Label and property required to drop a constraint.' + + # This endpoint is void, i.e. returns nothing: + # http://neo4j.com/docs/stable/rest-api-schema-constraints.html#rest-api-drop-constraint + # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + @http + method: 'DELETE' + path: "/db/data/schema/constraint/#{encodeURIComponent label}\ + /uniqueness/#{encodeURIComponent property}" + , cb + + # TODO: Legacy indexing diff --git a/lib-new/exports.coffee b/lib-new/exports.coffee index fd223e5..634dd0a 100644 --- a/lib-new/exports.coffee +++ b/lib-new/exports.coffee @@ -6,5 +6,6 @@ $(exports).extend Relationship: require './Relationship' Transaction: require './Transaction' Index: require './Index' + Constraint: require './Constraint' $(exports).extend require './errors' diff --git a/test-new/constraints._coffee b/test-new/constraints._coffee new file mode 100644 index 0000000..529945a --- /dev/null +++ b/test-new/constraints._coffee @@ -0,0 +1,260 @@ +# +# Tests for constraints, e.g. creating, retrieivng, and dropping constraints. +# In the process, also tests that constraints actually have an effect. +# + +{expect} = require 'chai' +fixtures = require './fixtures' +helpers = require './util/helpers' +neo4j = require '../' + + +## SHARED STATE + +{DB, TEST_LABEL} = fixtures + +[TEST_NODE_A, TEST_REL, TEST_NODE_B] = [] + +# Important: we generate a random prop to guarantee no same constraint yet. +# We set the value to the node's ID, to ensure uniqueness. +TEST_PROP = "constraint_#{helpers.getRandomStr()}" + +# These are the constraints that exist before this test suite runs... +ORIG_CONSTRAINTS_ALL = null # ...Across all labels +ORIG_CONSTRAINTS_LABEL = null # ...On our TEST_LABEL + +# And this is the constraint we create: +TEST_CONSTRAINT = null + + +## HELPERS + +expectConstraint = (constraint, label, property) -> + expect(constraint).to.be.an.instanceOf neo4j.Constraint + expect(constraint.label).to.equal label if label + expect(constraint.property).to.equal property if property + +expectConstraints = (constraints, label) -> + expect(constraints).to.be.an 'array' + for constraint in constraints + expectConstraint constraint, label + +violateConstraint = (_) -> + # Do this in a transaction, so that we don't actually persist: + tx = DB.beginTransaction() + + try + tx.cypher + query: """ + START n = node({idB}) + SET n.#{TEST_PROP} = {idA} + """ + params: + idA: TEST_NODE_A._id + idB: TEST_NODE_B._id + , _ + + # Not technically needed, but prevent Neo4j from waiting up to a minute for + # the transaction to expire in case of any errors: + finally + tx.rollback _ + + +## TESTS + +describe 'Constraints', -> + + # IMPORTANT: Mocha requires all test steps to be at the same nesting level + # for them to all execute in order. Hence "setup" and "teardown" wrappers. + + describe '(setup)', -> + + it '(create test nodes)', (_) -> + [TEST_NODE_A, TEST_REL, TEST_NODE_B] = + fixtures.createTestGraph module, 2, _ + + it '(set test properties)', (_) -> + DB.cypher + query: """ + START n = node({ids}) + SET n.#{TEST_PROP} = ID(n) + """ + params: + ids: [TEST_NODE_A._id, TEST_NODE_B._id] + , _ + + # Update our local instances too: + TEST_NODE_A.properties[TEST_PROP] = TEST_NODE_A._id + TEST_NODE_B.properties[TEST_PROP] = TEST_NODE_B._id + + + describe '(before constraint created)', -> + + it 'should support listing all constraints', (_) -> + constraints = DB.getConstraints _ + expectConstraints constraints + + ORIG_CONSTRAINTS_ALL = constraints + + it 'should support listing constraints for a particular label', (_) -> + constraints = DB.getConstraints TEST_LABEL, _ + expectConstraints constraints, TEST_LABEL + + ORIG_CONSTRAINTS_LABEL = constraints + + it 'should support querying for specific constraint', (_) -> + bool = DB.hasConstraint + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal false + + it '(verify constraint doesn’t exist yet)', (_) -> + # This shouldn't throw an error: + violateConstraint _ + + it 'should support creating constraint', (_) -> + constraint = DB.createConstraint + label: TEST_LABEL + property: TEST_PROP + , _ + + expectConstraint constraint, TEST_LABEL, TEST_PROP + + TEST_CONSTRAINT = constraint + + + describe '(after constraint created)', -> + + it '(verify by re-listing all constraints)', (_) -> + constraints = DB.getConstraints _ + expect(constraints).to.have.length ORIG_CONSTRAINTS_ALL.length + 1 + expect(constraints).to.contain TEST_CONSTRAINT + + it '(verify by re-listing constraints for test label)', (_) -> + constraints = DB.getConstraints TEST_LABEL, _ + expect(constraints).to.have.length ORIG_CONSTRAINTS_LABEL.length + 1 + expect(constraints).to.contain TEST_CONSTRAINT + + it '(verify by re-querying specific test constraint)', (_) -> + bool = DB.hasConstraint + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal true + + it '(verify with test query)', (done) -> + violateConstraint (err) -> + expect(err).to.exist() + + helpers.expectError err, + 'ClientError', 'Schema', 'ConstraintViolation', + "Node #{TEST_NODE_A._id} already exists + with label #{TEST_LABEL} + and property \"#{TEST_PROP}\"=[#{TEST_NODE_A._id}]" + + done() + + it 'should throw on create of already-created constraint', (done) -> + DB.createConstraint + label: TEST_LABEL + property: TEST_PROP + , (err, constraint) -> + expMessage = "Label '#{TEST_LABEL}' and property '#{TEST_PROP}' + already have a unique constraint defined on them." + + # Neo4j 2.2 returns a proper new-style error object for this + # case, but previous versions return an old-style error. + try + helpers.expectError err, 'ClientError', 'Schema', + 'ConstraintAlreadyExists', expMessage + catch assertionErr + # Check for the older case, but in case it fails, + # throw the original assertion error, not a new one. + try + helpers.expectOldError err, 409, + 'ConstraintViolationException', + 'org.neo4j.graphdb.ConstraintViolationException', + expMessage + catch doubleErr + throw assertionErr + + expect(constraint).to.not.exist() + done() + + it 'should support dropping constraint', (_) -> + DB.dropConstraint + label: TEST_LABEL + property: TEST_PROP + , _ + + + describe '(after constraint dropped)', -> + + it '(verify by re-listing all constraints)', (_) -> + constraints = DB.getConstraints _ + expect(constraints).to.eql ORIG_CONSTRAINTS_ALL + + it '(verify by re-listing constraints for test label)', (_) -> + constraints = DB.getConstraints TEST_LABEL, _ + expect(constraints).to.eql ORIG_CONSTRAINTS_LABEL + + it '(verify by re-querying specific test constraint)', (_) -> + bool = DB.hasConstraint + label: TEST_LABEL + property: TEST_PROP + , _ + + expect(bool).to.equal false + + it 'should throw on drop of already-dropped constraint', (done) -> + DB.dropConstraint + label: TEST_LABEL + property: TEST_PROP + , (err) -> + helpers.expectHttpError err, 404 + done() + + + describe '(misc)', -> + + it 'should require both label and property to query specific constraint', -> + for fn in [ + -> DB.hasConstraint null, -> + -> DB.hasConstraint '', -> + -> DB.hasConstraint {}, -> + -> DB.hasConstraint TEST_LABEL, -> + -> DB.hasConstraint {label: TEST_LABEL}, -> + -> DB.hasConstraint {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + it 'should require both label and property to create constraint', -> + for fn in [ + -> DB.createConstraint null, -> + -> DB.createConstraint '', -> + -> DB.createConstraint {}, -> + -> DB.createConstraint TEST_LABEL, -> + -> DB.createConstraint {label: TEST_LABEL}, -> + -> DB.createConstraint {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + it 'should require both label and property to drop constraint', -> + for fn in [ + -> DB.dropConstraint null, -> + -> DB.dropConstraint '', -> + -> DB.dropConstraint {}, -> + -> DB.dropConstraint TEST_LABEL, -> + -> DB.dropConstraint {label: TEST_LABEL}, -> + -> DB.dropConstraint {property: TEST_PROP}, -> + ] + expect(fn).to.throw TypeError, /label and property required/i + + + describe '(teardown)', -> + + it '(delete test nodes)', (_) -> + fixtures.deleteTestGraph module, _ From db4d6c86fc3dd7197fe32c8a5e7c0a4d10246c42 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 12 Mar 2015 00:03:37 -0400 Subject: [PATCH 061/121] v2 / Misc: bump package version to RC1! --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index cc3975f..587dc1a 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-alpha5", + "version": "2.0.0-RC1", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From 0f497550f6fefe62b97a106f1c375e0164b6c785 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 25 Mar 2015 23:17:32 -0400 Subject: [PATCH 062/121] v2 / Tests: update for Neo4j 2.2 GA fixes. --- .travis.yml | 2 +- lib-new/errors.coffee | 2 +- test-new/cypher._coffee | 6 ++++-- test-new/http._coffee | 2 +- test-new/transactions._coffee | 18 +++++++++--------- test-new/util/helpers._coffee | 25 ------------------------- 6 files changed, 16 insertions(+), 39 deletions(-) diff --git a/.travis.yml b/.travis.yml index 749ba0a..a6f554c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,7 +10,7 @@ jdk: env: # test across multiple versions of Neo4j: - - NEO4J_VERSION="2.2.0-RC01" + - NEO4J_VERSION="2.2.0" - NEO4J_VERSION="2.1.7" - NEO4J_VERSION="2.0.4" diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 42e1efd..516d2dd 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -51,7 +51,7 @@ class @Error extends Error # appropriate Error instance for it. # @_fromObject: (obj) -> - # NOTE: Neo4j seems to return both `stackTrace` and `stacktrace`. + # NOTE: Neo4j 2.2 seems to return both `stackTrace` and `stacktrace`. # https://github.com/neo4j/neo4j/issues/4145#issuecomment-78203290 # Normalizing to consistent `stackTrace` before we parse below! if obj.stacktrace? and not obj.stackTrace? diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index 717096a..fccf5fd 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -91,7 +91,8 @@ describe 'GraphDatabase::cypher', -> it 'should properly parse and throw Neo4j errors', (done) -> DB.cypher 'RETURN {foo}', (err, results) -> expect(err).to.exist() - helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' # Whether `results` are returned or not depends on the error; # Neo4j will return an array if the query could be executed, @@ -250,7 +251,8 @@ describe 'GraphDatabase::cypher', -> ] , (err, results) -> expect(err).to.exist() - helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' # NOTE: With batching, we *do* return any results that we # received before the error, in case of an open transaction. diff --git a/test-new/http._coffee b/test-new/http._coffee index b61d77d..5859fed 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -82,7 +82,7 @@ describe 'GraphDatabase::http', -> expect(body).to.not.exist() # TEMP: Neo4j 2.2 responds here with a new-style error object, - # but it's currently a `DatabaseError` in 2.2.0-RC01. + # but it's currently a `DatabaseError` in 2.2.0. # https://github.com/neo4j/neo4j/issues/4145 try helpers.expectOldError err, 404, 'NodeNotFoundException', diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 9cdd49f..0f00076 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -320,7 +320,8 @@ describe 'Transactions', -> idA: TEST_NODE_A._id , (err, results) => expect(err).to.exist() - helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'open' @@ -383,7 +384,8 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'rolled back' @@ -462,17 +464,14 @@ describe 'Transactions', -> expect(tx.state).to.equal 'open' # For precision, implementing this step without Streamline. - dbErred = null do (cont=_) => tx.cypher 'RETURN {foo}', (err, results) => expect(err).to.exist() - dbErred = helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() - # TEMP: Because Neo4j 2.2.0-RC01 incorrectly throws a `DatabaseError` - # for the above error (see `expectParameterMissingError` for details), - # the transaction does get rolled back. - expect(tx.state).to.equal (if dbErred then 'rolled back' else 'open') + expect(tx.state).to.equal 'open' it 'should properly handle fatal client errors on an auto-commit first query', (_) -> @@ -486,7 +485,8 @@ describe 'Transactions', -> commit: true , (err, results) => expect(err).to.exist() - helpers.expectParameterMissingError err + helpers.expectError err, 'ClientError', 'Statement', + 'ParameterMissing', 'Expected a parameter named foo' cont() expect(tx.state).to.equal 'rolled back' diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee index 934f4e4..825c95a 100644 --- a/test-new/util/helpers._coffee +++ b/test-new/util/helpers._coffee @@ -129,31 +129,6 @@ neo4j = require '../../' /// -# -# TEMP: Neo4j 2.2.0-RC01 incorrectly classifies `ParameterMissing` errors as -# `DatabaseError` rather than `ClientError`: -# https://github.com/neo4j/neo4j/issues/4144 -# -# Returns whether we did have to account for this bug or not. -# -@expectParameterMissingError = (err) => - try - @expectError err, 'ClientError', 'Statement', 'ParameterMissing', - 'Expected a parameter named foo' - return false - - catch assertionErr - # Check for the Neo4j 2.2.0-RC01 case, but if it's not, - # throw the original assertion error, not a new one. - try - @expectError err, 'DatabaseError', 'Statement', 'ExecutionFailure', - 'org.neo4j.graphdb.QueryExecutionException: - Expected a parameter named foo' - return true - - throw assertionErr - - # # Returns a random string. # From 0dbe4af249f4baa6ae20d56583dabad5ad9a0fe1 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 25 Mar 2015 23:33:32 -0400 Subject: [PATCH 063/121] v2 / Tests: update auth tests for Neo4j 2.2 GA; and simplify. --- test-new/_auth._coffee | 61 +++++++++--------------------------------- 1 file changed, 12 insertions(+), 49 deletions(-) diff --git a/test-new/_auth._coffee b/test-new/_auth._coffee index 251e029..5d49b22 100644 --- a/test-new/_auth._coffee +++ b/test-new/_auth._coffee @@ -18,11 +18,6 @@ helpers = require './util/helpers' neo4j = require '../' -## CONSTANTS - -AUTH_ERROR_CODE = 'Neo.ClientError.Security.AuthorizationFailed' - - ## SHARED STATE {DB} = fixtures @@ -46,25 +41,6 @@ disable = (reason) -> # TODO: It'd be nice if we could support bailing on *all* suites, # in case of an auth error, e.g. bad credentials. -tryGetDbVersion = (_) -> - try - fixtures.queryDbVersion _ - fixtures.DB_VERSION_NUM - catch err - NaN # since a boolean comparison of NaN with any number is false - -# TODO: De-duplicate this kind of helper between our test suites: -expectAuthError = (err, message) -> - expect(err).to.be.an.instanceof neo4j.ClientError - expect(err.name).to.equal 'neo4j.ClientError' - expect(err.message).to.equal "[#{AUTH_ERROR_CODE}] #{message}" - - expect(err.stack).to.contain '\n' - expect(err.stack.split('\n')[0]).to.equal "#{err.name}: #{err.message}" - - expect(err.neo4j).to.be.an 'object' - expect(err.neo4j).to.eql {code: AUTH_ERROR_CODE, message} - ## TESTS @@ -77,31 +53,15 @@ describe 'Auth', -> return disable 'Auth creds unspecified' # Querying user status (what this check method does) fails both when - # auth is unavailable (e.g. Neo4j 2.1) and when it's disabled. - # https://mix.fiftythree.com/aseemk/2471430 - # NOTE: This might need updating if the disabled 500 is fixed. - # https://github.com/neo4j/neo4j/issues/4138 + # auth is unavailable (Neo4j 2.1-) and when it's disabled (Neo4j 2.2+). try DB.checkPasswordChangeNeeded _ - - # If this worked, auth is available and enabled. - return - catch err - # If we're right that this means auth is unavailable or disabled, - # we can verify that by querying the Neo4j version. - dbVersion = tryGetDbVersion _ - - if (err instanceof neo4j.ClientError) and - (err.message.match /^404 /) and (dbVersion < 2.2) - return disable 'Neo4j <2.2 detected' - - if (err instanceof neo4j.DatabaseError) and - (err.message.match /^500 /) and (dbVersion >= 2.2) - return disable 'Neo4j auth appears disabled' - - disable 'Error checking auth' - throw err + if err instanceof neo4j.ClientError and err.message.match /^404 / + disable 'Auth disabled or unavailable' + else + disable 'Error checking auth' + throw err it 'should fail when auth is required but not set', (done) -> db = new neo4j.GraphDatabase @@ -112,7 +72,8 @@ describe 'Auth', -> # rejects calls when no auth is set. db.http '/db/data/', (err, data) -> expect(err).to.exist() - expectAuthError err, 'No authorization header supplied.' + helpers.expectError err, 'ClientError', 'Security', + 'AuthorizationFailed', 'No authorization header supplied.' expect(data).to.not.exist() done() @@ -125,7 +86,8 @@ describe 'Auth', -> db.checkPasswordChangeNeeded (err, bool) -> expect(err).to.exist() - expectAuthError err, 'Invalid username or password.' + helpers.expectError err, 'ClientError', 'Security', + 'AuthorizationFailed', 'Invalid username or password.' expect(bool).to.not.exist() done() @@ -138,7 +100,8 @@ describe 'Auth', -> db.checkPasswordChangeNeeded (err, bool) -> expect(err).to.exist() - expectAuthError err, 'Invalid username or password.' + helpers.expectError err, 'ClientError', 'Security', + 'AuthorizationFailed', 'Invalid username or password.' expect(bool).to.not.exist() done() From e383c8f45bd76648285ed8f54dfd11c6f6eb2d5c Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 26 Mar 2015 13:10:48 -0400 Subject: [PATCH 064/121] v2 / HTTP: more robust headers; clone before modifying. --- lib-new/GraphDatabase.coffee | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 3ee2018..555905d 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -19,7 +19,6 @@ module.exports = class GraphDatabase # Default HTTP headers: headers: 'User-Agent': "node-neo4j/#{lib.version}" - 'X-Stream': 'true' constructor: (opts={}) -> if typeof opts is 'string' @@ -40,9 +39,15 @@ module.exports = class GraphDatabase # We also normalize any given auth to an object or null: @auth = _normalizeAuth @auth ? uri.auth - # TODO: Do we want to special-case User-Agent? Blacklist X-Stream? + # Extend the given headers with our defaults, but clone first: + # TODO: Do we want to special-case User-Agent? Or reject if includes + # reserved headers like Accept, Content-Type, X-Stream? @headers or= {} - $(@headers).defaults @constructor::headers + @headers = $(@headers) + .chain() + .clone() + .defaults @constructor::headers + .value() ## HTTP @@ -59,6 +64,18 @@ module.exports = class GraphDatabase method or= 'GET' headers or= {} + # Extend the given headers, both with both required and optional + # defaults, but do so without modifying the input object: + headers = $(headers) + .chain() + .clone() + .defaults @headers # These headers can be overridden... + .extend # ...while these can't. + 'Accept': 'application/json' + 'Content-Type': 'application/json' + 'X-Stream': 'true' + .value() + # TODO: Would be good to test custom proxy and agent, but difficult. # Same with Neo4j returning gzipped responses (e.g. through an LB). req = Request @@ -66,7 +83,7 @@ module.exports = class GraphDatabase url: URL.resolve @url, path proxy: @proxy auth: @auth - headers: $(headers).defaults @headers + headers: headers agent: @agent json: body ? true gzip: true # This is only for responses: decode if gzipped. From b74b0812f9f0442f45a7373f3a7d5fa4eda4765a Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 26 Mar 2015 21:06:48 -0400 Subject: [PATCH 065/121] v2 / HTTP: clarify streaming API; expand and improve tests. --- API_v2.md | 5 ++ test-new/http._coffee | 135 ++++++++++++++++++++++++++++++++---------- 2 files changed, 110 insertions(+), 30 deletions(-) diff --git a/API_v2.md b/API_v2.md index 14aa53f..8fdda10 100644 --- a/API_v2.md +++ b/API_v2.md @@ -113,6 +113,11 @@ response data can be streamed. [`ClientRequest`]: http://nodejs.org/api/http.html#http_class_http_clientrequest [`IncomingMessage`]: http://nodejs.org/api/http.html#http_http_incomingmessage +*Important: if this is a write request (`POST`, `PUT`, `DELETE`, etc.), +you may still pass a request `body` for convenience, but if you don't, +it's your responsibility to pass `Content-Type` and `Content-Length` headers, +as well as call `req.end()` to finish the request.* + In addition, if a callback is given, it will be called with the final result. By default, this result will be the HTTP response body (parsed as JSON), with nodes and relationships transformed to diff --git a/test-new/http._coffee b/test-new/http._coffee index 5859fed..c399f38 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -14,6 +14,7 @@ neo4j = require '../' ## CONSTANTS FAKE_JSON_PATH = "#{__dirname}/fixtures/fake.json" +FAKE_JSON = require FAKE_JSON_PATH ## SHARED STATE @@ -26,14 +27,30 @@ FAKE_JSON_PATH = "#{__dirname}/fixtures/fake.json" ## HELPERS # -# Asserts that the given object is a proper HTTP client response with the given -# status code. +# Asserts that the given object is a proper HTTP client request, +# set to the given method and path, and optionally including the given headers. # -expectResponse = (resp, statusCode) -> +expectRequest = (req, method, path, headers={}) -> + expect(req).to.be.an.instanceOf http.ClientRequest + expect(req.method).to.equal method + expect(req.path).to.equal path + + # Special-case for headers since they're stored differently: + for name, val of headers + expect(req.getHeader name).to.equal val + +# +# Asserts that the given object is a proper HTTP client response, +# with the given status code, and by default, a JSON Content-Type. +# +expectResponse = (resp, statusCode, json=true) -> expect(resp).to.be.an.instanceOf http.IncomingMessage expect(resp.statusCode).to.equal statusCode expect(resp.headers).to.be.an 'object' + if json + expect(resp.headers['content-type']).to.match /^application\/json\b/ + # # Asserts that the given object is the root Neo4j object. # @@ -41,6 +58,30 @@ expectNeo4jRoot = (body) -> expect(body).to.be.an 'object' expect(body).to.have.keys 'data', 'management' +# +# Streams the given response's JSON body, and calls either the error callback +# (convenient for failing tests) or the success callback (with the parsed body). +# +streamResponse = (resp, cbErr, cbBody) -> + body = null + + resp.setEncoding 'utf8' + + resp.on 'error', cbErr + resp.on 'close', -> cbErr new Error 'Response closed!' + + resp.on 'data', (str) -> + body ?= '' + body += str + + resp.on 'end', -> + try + body = JSON.parse body + catch err + return cbErr err + + cbBody body + ## TESTS @@ -109,7 +150,6 @@ describe 'GraphDatabase::http', -> , _ expectResponse resp, 200 - expect(resp.headers['content-type']).to.match /// ^application/json\b /// expectNeo4jRoot resp.body it 'should not throw 4xx errors for raw responses', (_) -> @@ -119,7 +159,7 @@ describe 'GraphDatabase::http', -> raw: true , _ - expectResponse resp, 405 # Method Not Allowed + expectResponse resp, 405, false # Method Not Allowed, no JSON body it 'should throw native errors always', (done) -> db = new neo4j.GraphDatabase 'http://idontexist.foobarbaz.nodeneo4j' @@ -144,11 +184,67 @@ describe 'GraphDatabase::http', -> done() - it 'should support streaming', (done) -> + it 'should support streaming responses', (done) -> + opts = + method: 'GET' + path: '/db/data/node/-1' + + req = DB.http opts + + expectRequest req, opts.method, opts.path + + # Native errors are emitted on this request, so fail-fast if any: + req.on 'error', done + + # Since this is a GET, no need for us to call `req.end()` manually. + # When the response is received, stream down its JSON and verify: + req.on 'response', (resp) -> + expectResponse resp, 404 + streamResponse resp, done, (body) -> + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'NodeNotFoundException' + expect(body.message).to.equal ' + Cannot find node with id [-1] in database.' + expect(body.stacktrace).to.be.an 'array' + + done() + + it 'should support streaming responses, even if not requests', (done) -> + opts = + method: 'POST' + path: '/db/data/node' + body: FAKE_JSON + + req = DB.http opts + + # TODO: Should we also assert that the request has automatically added + # Content-Type and Content-Length headers? Not technically required? + expectRequest req, opts.method, opts.path + + # Native errors are emitted on this request, so fail-fast if any: + req.on 'error', done + + # Since we supplied a body, no need for us to call `req.end()` manually. + # When the response is received, stream down its JSON and verify: + req.on 'response', (resp) -> + expectResponse resp, 400 + streamResponse resp, done, (body) -> + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'PropertyValueException' + expect(body.message).to.equal 'Could not set property "object", + unsupported type: {foo={bar=baz}}' + expect(body.stacktrace).to.be.an 'array' + + done() + + it 'should support streaming both requests and responses', (done) -> opts = method: 'POST' path: '/db/data/node' headers: + 'Content-Type': 'application/json' # NOTE: It seems that Neo4j needs an explicit Content-Length, # at least for requests to this `POST /db/data/node` endpoint. 'Content-Length': (fs.statSync FAKE_JSON_PATH).size @@ -158,25 +254,13 @@ describe 'GraphDatabase::http', -> req = DB.http opts - expect(req).to.be.an.instanceOf http.ClientRequest - expect(req.method).to.equal opts.method - expect(req.path).to.equal opts.path - - # Special-case for headers since they're stored differently: - for name, val of opts.headers - expect(req.getHeader name).to.equal val + expectRequest req, opts.method, opts.path, opts.headers # Native errors are emitted on this request, so fail-fast if any: req.on 'error', done # Now stream some fake JSON to the request: - # TODO: Why doesn't this work? - # fs.createReadStream(FAKE_JSON_PATH).pipe req - # TEMP: Instead, we have to manually pipe: - readStream = fs.createReadStream FAKE_JSON_PATH - readStream.on 'error', done - readStream.on 'data', (chunk) -> req.write chunk - readStream.on 'end', -> req.end() + fs.createReadStream(FAKE_JSON_PATH).pipe req # Verify that the request fully waits for our stream to finish # before returning a response: @@ -187,16 +271,7 @@ describe 'GraphDatabase::http', -> req.on 'response', (resp) -> expect(finished).to.be.true() expectResponse resp, 400 - - resp.setEncoding 'utf8' - body = '' - - resp.on 'data', (str) -> body += str - resp.on 'error', done - resp.on 'close', -> done new Error 'Response closed!' - resp.on 'end', -> - body = JSON.parse body - + streamResponse resp, done, (body) -> # Simplified error parsing; just verifying stream: expect(body).to.be.an 'object' expect(body.exception).to.equal 'PropertyValueException' From 2403662d5e1a94de977e695d1bd838d6fff01437 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 26 Mar 2015 13:12:32 -0400 Subject: [PATCH 066/121] v2 / HTTP: switch to native module; hopefully fixes streaming error? But importantly, this regresses in functionality: no proxy, no gzip. --- lib-new/GraphDatabase.coffee | 100 ++++++++++++++++++++++------------- 1 file changed, 64 insertions(+), 36 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 555905d..e492202 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -2,6 +2,8 @@ $ = require 'underscore' assert = require 'assert' Constraint = require './Constraint' {Error} = require './errors' +http = require 'http' +https = require 'https' Index = require './Index' lib = require '../package.json' Node = require './Node' @@ -31,13 +33,13 @@ module.exports = class GraphDatabase # Process auth, whether through option or URL creds or both. # Option takes precedence, and we clear the URL creds if option given. - uri = URL.parse @url - if uri.auth and @auth? - delete uri.auth - @url = URL.format uri + @uri = URL.parse @url + if @uri.auth and @auth? + delete @uri.auth + @url = URL.format @uri # We also normalize any given auth to an object or null: - @auth = _normalizeAuth @auth ? uri.auth + @auth = _normalizeAuth @auth ? @uri.auth # Extend the given headers with our defaults, but clone first: # TODO: Do we want to special-case User-Agent? Or reject if includes @@ -72,45 +74,71 @@ module.exports = class GraphDatabase .defaults @headers # These headers can be overridden... .extend # ...while these can't. 'Accept': 'application/json' - 'Content-Type': 'application/json' 'X-Stream': 'true' .value() + if body + body = JSON.stringify body + $(headers).extend + 'Content-Type': 'application/json' + 'Content-Length': (new Buffer body).length + else if cb + $(headers).extend + 'Content-Length': 0 + # TODO: Would be good to test custom proxy and agent, but difficult. # Same with Neo4j returning gzipped responses (e.g. through an LB). - req = Request + net = require @uri.protocol[...-1] # E.g. `http` or `https` + req = net.request method: method - url: URL.resolve @url, path - proxy: @proxy - auth: @auth + hostname: @uri.hostname + port: @uri.port + path: path + # proxy: @proxy # TODO + auth: if @auth then "#{@auth.username}:#{@auth.password}" headers: headers agent: @agent - json: body ? true - gzip: true # This is only for responses: decode if gzipped. - - # Important: only pass a callback to Request if a callback was passed - # to us. This prevents Request from doing unnecessary JSON parse work - # if the caller prefers to stream the response instead of buffer it. - , cb and (err, resp) => - if err - # TODO: Do we want to wrap or modify native errors? - return cb err - - if raw - # TODO: Do we want to return our own Response object? - return cb null, resp - - if err = Error._fromResponse resp - return cb err - - cb null, _transform resp.body - - # Instead of leaking our (third-party) Request instance, make sure to - # explicitly return only its internal native ClientRequest instance. - # https://github.com/request/request/blob/v2.53.1/request.js#L904 - # This is only populated when the request is `start`ed, so `start` it! - req.start() - req.req + keepAlive: true + # gzip: true # This is only for responses: decode if gzipped. + # TODO + + # If a request body was given, go ahead send it and end the request: + if body + req.write body + req.end() + + # Also go ahead end the request if this is a read request: + if method.toUpperCase() in ['GET', 'HEAD', 'OPTIONS'] + req.end() + + # Important: only buffer the response data if a callback was given. + # (And go ahead end the request then too, for the no-body case.) + # If no callback was given, let the caller stream the response data, + # and also support them streaming the request body then. + # (Note: no harm in calling `req.end` twice.) + if cb + req.end() + req.on 'error', cb + req.on 'response', (resp) -> + resp.setEncoding 'utf8' + + resp.on 'data', (chunk) -> + resp.body ?= '' + resp.body += chunk + + resp.on 'end', -> + try + resp.body = JSON.parse resp.body + + if raw + return cb null, resp + + if err = Error._fromResponse resp + return cb err + + cb null, _transform resp.body + + req ## AUTH From 90efd313b2f117a717ebe08d5e33584c96b5332e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 26 Mar 2015 21:29:00 -0400 Subject: [PATCH 067/121] v2 / HTTP: revert switch to native module. This reverts commit 2750749b2140360d7cca135dfd0fbb65f69bc7f1. --- lib-new/GraphDatabase.coffee | 100 +++++++++++++---------------------- 1 file changed, 36 insertions(+), 64 deletions(-) diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index e492202..555905d 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -2,8 +2,6 @@ $ = require 'underscore' assert = require 'assert' Constraint = require './Constraint' {Error} = require './errors' -http = require 'http' -https = require 'https' Index = require './Index' lib = require '../package.json' Node = require './Node' @@ -33,13 +31,13 @@ module.exports = class GraphDatabase # Process auth, whether through option or URL creds or both. # Option takes precedence, and we clear the URL creds if option given. - @uri = URL.parse @url - if @uri.auth and @auth? - delete @uri.auth - @url = URL.format @uri + uri = URL.parse @url + if uri.auth and @auth? + delete uri.auth + @url = URL.format uri # We also normalize any given auth to an object or null: - @auth = _normalizeAuth @auth ? @uri.auth + @auth = _normalizeAuth @auth ? uri.auth # Extend the given headers with our defaults, but clone first: # TODO: Do we want to special-case User-Agent? Or reject if includes @@ -74,71 +72,45 @@ module.exports = class GraphDatabase .defaults @headers # These headers can be overridden... .extend # ...while these can't. 'Accept': 'application/json' + 'Content-Type': 'application/json' 'X-Stream': 'true' .value() - if body - body = JSON.stringify body - $(headers).extend - 'Content-Type': 'application/json' - 'Content-Length': (new Buffer body).length - else if cb - $(headers).extend - 'Content-Length': 0 - # TODO: Would be good to test custom proxy and agent, but difficult. # Same with Neo4j returning gzipped responses (e.g. through an LB). - net = require @uri.protocol[...-1] # E.g. `http` or `https` - req = net.request + req = Request method: method - hostname: @uri.hostname - port: @uri.port - path: path - # proxy: @proxy # TODO - auth: if @auth then "#{@auth.username}:#{@auth.password}" + url: URL.resolve @url, path + proxy: @proxy + auth: @auth headers: headers agent: @agent - keepAlive: true - # gzip: true # This is only for responses: decode if gzipped. - # TODO - - # If a request body was given, go ahead send it and end the request: - if body - req.write body - req.end() - - # Also go ahead end the request if this is a read request: - if method.toUpperCase() in ['GET', 'HEAD', 'OPTIONS'] - req.end() - - # Important: only buffer the response data if a callback was given. - # (And go ahead end the request then too, for the no-body case.) - # If no callback was given, let the caller stream the response data, - # and also support them streaming the request body then. - # (Note: no harm in calling `req.end` twice.) - if cb - req.end() - req.on 'error', cb - req.on 'response', (resp) -> - resp.setEncoding 'utf8' - - resp.on 'data', (chunk) -> - resp.body ?= '' - resp.body += chunk - - resp.on 'end', -> - try - resp.body = JSON.parse resp.body - - if raw - return cb null, resp - - if err = Error._fromResponse resp - return cb err - - cb null, _transform resp.body - - req + json: body ? true + gzip: true # This is only for responses: decode if gzipped. + + # Important: only pass a callback to Request if a callback was passed + # to us. This prevents Request from doing unnecessary JSON parse work + # if the caller prefers to stream the response instead of buffer it. + , cb and (err, resp) => + if err + # TODO: Do we want to wrap or modify native errors? + return cb err + + if raw + # TODO: Do we want to return our own Response object? + return cb null, resp + + if err = Error._fromResponse resp + return cb err + + cb null, _transform resp.body + + # Instead of leaking our (third-party) Request instance, make sure to + # explicitly return only its internal native ClientRequest instance. + # https://github.com/request/request/blob/v2.53.1/request.js#L904 + # This is only populated when the request is `start`ed, so `start` it! + req.start() + req.req ## AUTH From c803c9bf90c35b9e73850bf330e83d6cfc77c2e3 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Fri, 27 Mar 2015 13:24:19 -0400 Subject: [PATCH 068/121] v2 / HTTP: embrace Request.js for streaming! --- API_v2.md | 16 +++--- lib-new/GraphDatabase.coffee | 16 ++---- test-new/http._coffee | 97 ++++++++++++++---------------------- 3 files changed, 48 insertions(+), 81 deletions(-) diff --git a/API_v2.md b/API_v2.md index 8fdda10..ff65ab1 100644 --- a/API_v2.md +++ b/API_v2.md @@ -105,19 +105,17 @@ function cb(err, body) {} var req = db.http({method, path, headers, body, raw}, cb); ``` -This method will immediately return a native HTTP [`ClientRequest`][], -to which request data can be streamed, and which emits a `'response'` event -yielding a native HTTP [`IncomingMessage`][], from which (raw chunks of) -response data can be streamed. +This method returns a [`Request.js`][] instance, which is a duplex stream +to and from which both request and response data can be streamed. +(`Request.js` provides a number of benefits over the native HTTP +[`ClientRequest`][] and [`IncomingMessage`][] classes, such as proxy support, +gzip decompression, simpler writes, and a single, unified `'error'` event.) + +[`Request.js`]: https://github.com/request/request [`ClientRequest`]: http://nodejs.org/api/http.html#http_class_http_clientrequest [`IncomingMessage`]: http://nodejs.org/api/http.html#http_http_incomingmessage -*Important: if this is a write request (`POST`, `PUT`, `DELETE`, etc.), -you may still pass a request `body` for convenience, but if you don't, -it's your responsibility to pass `Content-Type` and `Content-Length` headers, -as well as call `req.end()` to finish the request.* - In addition, if a callback is given, it will be called with the final result. By default, this result will be the HTTP response body (parsed as JSON), with nodes and relationships transformed to diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 555905d..0fe6ecb 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -71,14 +71,12 @@ module.exports = class GraphDatabase .clone() .defaults @headers # These headers can be overridden... .extend # ...while these can't. - 'Accept': 'application/json' - 'Content-Type': 'application/json' 'X-Stream': 'true' .value() # TODO: Would be good to test custom proxy and agent, but difficult. # Same with Neo4j returning gzipped responses (e.g. through an LB). - req = Request + Request method: method url: URL.resolve @url, path proxy: @proxy @@ -86,11 +84,12 @@ module.exports = class GraphDatabase headers: headers agent: @agent json: body ? true + encoding: 'utf8' gzip: true # This is only for responses: decode if gzipped. # Important: only pass a callback to Request if a callback was passed - # to us. This prevents Request from doing unnecessary JSON parse work - # if the caller prefers to stream the response instead of buffer it. + # to us. This prevents Request from buffering the response in memory + # (to parse JSON) if the caller prefers to stream the response instead. , cb and (err, resp) => if err # TODO: Do we want to wrap or modify native errors? @@ -105,13 +104,6 @@ module.exports = class GraphDatabase cb null, _transform resp.body - # Instead of leaking our (third-party) Request instance, make sure to - # explicitly return only its internal native ClientRequest instance. - # https://github.com/request/request/blob/v2.53.1/request.js#L904 - # This is only populated when the request is `start`ed, so `start` it! - req.start() - req.req - ## AUTH diff --git a/test-new/http._coffee b/test-new/http._coffee index c399f38..cdc3b46 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -9,6 +9,7 @@ fs = require 'fs' helpers = require './util/helpers' http = require 'http' neo4j = require '../' +Request = require 'request' ## CONSTANTS @@ -31,7 +32,7 @@ FAKE_JSON = require FAKE_JSON_PATH # set to the given method and path, and optionally including the given headers. # expectRequest = (req, method, path, headers={}) -> - expect(req).to.be.an.instanceOf http.ClientRequest + expect(req).to.be.an.instanceOf Request.Request expect(req.method).to.equal method expect(req.path).to.equal path @@ -59,22 +60,24 @@ expectNeo4jRoot = (body) -> expect(body).to.have.keys 'data', 'management' # -# Streams the given response's JSON body, and calls either the error callback -# (convenient for failing tests) or the success callback (with the parsed body). +# Streams the given Request response, and calls either the error callback +# (for failing tests) or the success callback (with the parsed JSON body). +# Also asserts that the response has the given status code. # -streamResponse = (resp, cbErr, cbBody) -> +streamRequestResponse = (req, statusCode, cbErr, cbBody) -> body = null - resp.setEncoding 'utf8' + req.on 'error', cbErr + req.on 'close', -> cbErr new Error 'Response closed!' - resp.on 'error', cbErr - resp.on 'close', -> cbErr new Error 'Response closed!' + req.on 'response', (resp) -> + expectResponse resp, statusCode - resp.on 'data', (str) -> + req.on 'data', (str) -> body ?= '' body += str - resp.on 'end', -> + req.on 'end', -> try body = JSON.parse body catch err @@ -193,22 +196,15 @@ describe 'GraphDatabase::http', -> expectRequest req, opts.method, opts.path - # Native errors are emitted on this request, so fail-fast if any: - req.on 'error', done + streamRequestResponse req, 404, done, (body) -> + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'NodeNotFoundException' + expect(body.message).to.equal ' + Cannot find node with id [-1] in database.' + expect(body.stacktrace).to.be.an 'array' - # Since this is a GET, no need for us to call `req.end()` manually. - # When the response is received, stream down its JSON and verify: - req.on 'response', (resp) -> - expectResponse resp, 404 - streamResponse resp, done, (body) -> - # Simplified error parsing; just verifying stream: - expect(body).to.be.an 'object' - expect(body.exception).to.equal 'NodeNotFoundException' - expect(body.message).to.equal ' - Cannot find node with id [-1] in database.' - expect(body.stacktrace).to.be.an 'array' - - done() + done() it 'should support streaming responses, even if not requests', (done) -> opts = @@ -222,22 +218,15 @@ describe 'GraphDatabase::http', -> # Content-Type and Content-Length headers? Not technically required? expectRequest req, opts.method, opts.path - # Native errors are emitted on this request, so fail-fast if any: - req.on 'error', done + streamRequestResponse req, 400, done, (body) -> + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'PropertyValueException' + expect(body.message).to.equal 'Could not set property "object", + unsupported type: {foo={bar=baz}}' + expect(body.stacktrace).to.be.an 'array' - # Since we supplied a body, no need for us to call `req.end()` manually. - # When the response is received, stream down its JSON and verify: - req.on 'response', (resp) -> - expectResponse resp, 400 - streamResponse resp, done, (body) -> - # Simplified error parsing; just verifying stream: - expect(body).to.be.an 'object' - expect(body.exception).to.equal 'PropertyValueException' - expect(body.message).to.equal 'Could not set property "object", - unsupported type: {foo={bar=baz}}' - expect(body.stacktrace).to.be.an 'array' - - done() + done() it 'should support streaming both requests and responses', (done) -> opts = @@ -256,30 +245,18 @@ describe 'GraphDatabase::http', -> expectRequest req, opts.method, opts.path, opts.headers - # Native errors are emitted on this request, so fail-fast if any: - req.on 'error', done - # Now stream some fake JSON to the request: fs.createReadStream(FAKE_JSON_PATH).pipe req - # Verify that the request fully waits for our stream to finish - # before returning a response: - finished = false - req.on 'finish', -> finished = true - - # When the response is received, stream down its JSON too: - req.on 'response', (resp) -> - expect(finished).to.be.true() - expectResponse resp, 400 - streamResponse resp, done, (body) -> - # Simplified error parsing; just verifying stream: - expect(body).to.be.an 'object' - expect(body.exception).to.equal 'PropertyValueException' - expect(body.message).to.equal 'Could not set property "object", - unsupported type: {foo={bar=baz}}' - expect(body.stacktrace).to.be.an 'array' - - done() + streamRequestResponse req, 400, done, (body) -> + # Simplified error parsing; just verifying stream: + expect(body).to.be.an 'object' + expect(body.exception).to.equal 'PropertyValueException' + expect(body.message).to.equal 'Could not set property "object", + unsupported type: {foo={bar=baz}}' + expect(body.stacktrace).to.be.an 'array' + + done() ## Object parsing: From e2fc513a293a83ef8ca7e9bd7aabc9aef4526f81 Mon Sep 17 00:00:00 2001 From: Brian Underwood Date: Fri, 10 Apr 2015 15:28:31 +0200 Subject: [PATCH 069/121] v2 / Docs: readme improvements from Neo4j team. --- README.md | 55 +++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index a7523d4..d866c1f 100644 --- a/README.md +++ b/README.md @@ -13,13 +13,18 @@ and `float: right` them in the Node-Neo4j header. (Admittedly, yucky markup.) # Node-Neo4j npm version -This is a Node.js driver for [Neo4j](http://neo4j.com/), a graph database. +This is a [Node.js][node.js] driver for [Neo4j][neo4j] via it's [REST API][neo4j-rest-api]. **This driver has undergone a complete rewrite for Neo4j v2.** It now *only* supports Neo4j 2.x — but it supports it really well. (If you're still on Neo4j 1.x, you can still use [node-neo4j v1](https://github.com/thingdom/node-neo4j/tree/v1).) +## What is Neo4j? + +Neo4j is a transactional, open-source graph database. A graph database manages data in a connected data structure, capable of representing any kind of data in a very accessible way. Information is stored in nodes and relationships connecting them, both of which can have arbitrary properties. To learn more visit [What is a Graph Database?][what-is-a-graph-database] + + @@ -33,12 +38,11 @@ Similarly, mention used in production by FiftyThree? --> npm install neo4j --save ``` - ## Usage ```js var neo4j = require('neo4j'); -var db = new neo4j.GraphDatabase('http://localhost:7474'); +var db = new neo4j.GraphDatabase('http://username:password@localhost:7474'); db.cypher({ query: 'MATCH (u:User {email: {email}}) RETURN u', @@ -75,6 +79,39 @@ Yields e.g.: } ``` +## Getting Help + +If you're having any issues you can first refer to the [API documentation][api-docs]. + +If you encounter any bugs or other issues, please file them in the +[issue tracker][issue-tracker]. + +We also now have a [Google Group][google-group]! +Post questions and participate in general discussions there. + +You can also [ask a question on StackOverflow][stackoverflow-ask] + + +## Neo4j version support + +| **Version** | **Ver 1.x** | **Ver 2.x** | +|-------------|--------------|-------------| +| 1.5-1.9 | Yes | No | +| 2.0 | Yes | Yes | +| 2.1 | Yes | Yes | +| 2.2 | No | Yes | + +## Neo4j feature support + +| **Feature** | **Ver 1.x** | **Ver 2.x** | +|----------------------|-------------|-------------| +| Auth | No | Yes | +| Remote Cypher | Yes | Yes | +| Transactions | No | No | +| High Availability | No | No | +| Embedded JVM support | No | No | + + Node.js is asynchronous, which means this library is too: most functions take @@ -154,16 +191,9 @@ See the [Changelog][changelog] for the full history of changes and releases. This library is licensed under the [Apache License, Version 2.0][license]. -## Feedback - -If you encounter any bugs or other issues, please file them in the -[issue tracker][issue-tracker]. - -We also now have a [Google Group][google-group]! -Post questions and participate in general discussions there. - [neo4j]: http://neo4j.org/ +[what-is-a-graph-database]: http://neo4j.com/developer/graph-database/ [node.js]: http://nodejs.org/ [neo4j-rest-api]: http://docs.neo4j.org/chunked/stable/rest-api.html @@ -172,7 +202,6 @@ Post questions and participate in general discussions there. [node-neo4j-template]: https://github.com/aseemk/node-neo4j-template [semver]: http://semver.org/ -[neo4j-getting-started]: http://wiki.neo4j.org/content/Getting_Started_With_Neo4j_Server [coffeescript]: http://coffeescript.org/ [streamline.js]: https://github.com/Sage/streamlinejs @@ -180,3 +209,5 @@ Post questions and participate in general discussions there. [issue-tracker]: https://github.com/thingdom/node-neo4j/issues [license]: http://www.apache.org/licenses/LICENSE-2.0.html [google-group]: https://groups.google.com/group/node-neo4j + +[stackoverflow-ask]: http://stackoverflow.com/questions/ask?tags=node.js,neo4j,thingdom From 0fc9e75c78db86f3e231c63838f0d05d4e04195c Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 12 May 2015 13:12:44 -0400 Subject: [PATCH 070/121] v2 / Misc: add CoffeeLint! And lint in CI. --- .travis.yml | 4 ++ coffeelint.json | 122 ++++++++++++++++++++++++++++++++++++++++++++++++ package.json | 2 + 3 files changed, 128 insertions(+) create mode 100644 coffeelint.json diff --git a/.travis.yml b/.travis.yml index a6f554c..cf5f334 100644 --- a/.travis.yml +++ b/.travis.yml @@ -31,3 +31,7 @@ before_install: branches: only: - master + +script: + - npm run lint + - npm test diff --git a/coffeelint.json b/coffeelint.json new file mode 100644 index 0000000..7ae969d --- /dev/null +++ b/coffeelint.json @@ -0,0 +1,122 @@ +{ + "arrow_spacing": { + "level": "ignore" + }, + "braces_spacing": { + "level": "ignore", + "spaces": 0, + "empty_object_spaces": 0 + }, + "camel_case_classes": { + "level": "error" + }, + "coffeescript_error": { + "level": "error" + }, + "colon_assignment_spacing": { + "level": "ignore", + "spacing": { + "left": 0, + "right": 0 + } + }, + "cyclomatic_complexity": { + "value": 10, + "level": "ignore" + }, + "duplicate_key": { + "level": "error" + }, + "empty_constructor_needs_parens": { + "level": "ignore" + }, + "ensure_comprehensions": { + "level": "warn" + }, + "indentation": { + "value": 4, + "level": "error" + }, + "line_endings": { + "level": "ignore", + "value": "unix" + }, + "max_line_length": { + "value": 80, + "level": "error", + "limitComments": true + }, + "missing_fat_arrows": { + "level": "ignore", + "is_strict": false + }, + "newlines_after_classes": { + "value": 3, + "level": "ignore" + }, + "no_backticks": { + "level": "error" + }, + "no_debugger": { + "level": "warn" + }, + "no_empty_functions": { + "level": "ignore" + }, + "no_empty_param_list": { + "level": "ignore" + }, + "no_implicit_braces": { + "level": "ignore", + "strict": true + }, + "no_implicit_parens": { + "strict": true, + "level": "ignore" + }, + "no_interpolation_in_single_quotes": { + "level": "ignore" + }, + "no_plusplus": { + "level": "ignore" + }, + "no_stand_alone_at": { + "level": "ignore" + }, + "no_tabs": { + "level": "error" + }, + "no_throwing_strings": { + "level": "error" + }, + "no_trailing_semicolons": { + "level": "error" + }, + "no_trailing_whitespace": { + "level": "error", + "allowed_in_comments": false, + "allowed_in_empty_lines": true + }, + "no_unnecessary_double_quotes": { + "level": "ignore" + }, + "no_unnecessary_fat_arrows": { + "level": "warn" + }, + "non_empty_constructor_needs_parens": { + "level": "ignore" + }, + "prefer_english_operator": { + "level": "ignore", + "doubleNotLevel": "ignore" + }, + "space_operators": { + "level": "ignore" + }, + "spacing_after_comma": { + "level": "ignore" + }, + "transform_messes_up_line_numbers": { + "level": "warn" + } +} diff --git a/package.json b/package.json index 587dc1a..cc12d22 100644 --- a/package.json +++ b/package.json @@ -15,6 +15,7 @@ "devDependencies": { "chai": "^1.9.2", "coffee-script": "1.8.x", + "coffeelint": "^1.9.7", "mocha": "^2.0.1", "streamline": "^0.10.16" }, @@ -24,6 +25,7 @@ "scripts": { "build": "coffee -m -c lib-new/", "clean": "rm -f lib-new/*.{js,map}", + "lint": "git ls-files | grep coffee$ | grep -v '\\-old/' | xargs coffeelint", "prepublish": "npm run build", "postpublish": "npm run clean", "test": "mocha test-new" From cad1443a7c7edf3ee9a3f0003f365e82c4cb2315 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 26 May 2015 12:02:49 -0400 Subject: [PATCH 071/121] v2 / Misc: improve CoffeeLint rules, and clean up code. --- coffeelint.json | 73 ++++++++++++++++++++------------- lib-new/GraphDatabase.coffee | 16 ++++++-- lib-new/Transaction.coffee | 5 +++ test-new/constraints._coffee | 41 +++++++++--------- test-new/constructor._coffee | 4 +- test-new/cypher._coffee | 13 +++--- test-new/fixtures/index._coffee | 4 +- test-new/http._coffee | 4 +- test-new/indexes._coffee | 38 +++++++++-------- test-new/transactions._coffee | 8 ++++ test-new/util/helpers._coffee | 2 +- 11 files changed, 125 insertions(+), 83 deletions(-) diff --git a/coffeelint.json b/coffeelint.json index 7ae969d..8b5fa95 100644 --- a/coffeelint.json +++ b/coffeelint.json @@ -1,9 +1,9 @@ { "arrow_spacing": { - "level": "ignore" + "level": "error" }, "braces_spacing": { - "level": "ignore", + "level": "error", "spaces": 0, "empty_object_spaces": 0 }, @@ -14,31 +14,34 @@ "level": "error" }, "colon_assignment_spacing": { - "level": "ignore", + "level": "error", "spacing": { "left": 0, - "right": 0 + "right": 1 } }, "cyclomatic_complexity": { - "value": 10, - "level": "ignore" + "value": 15, + "level": "warn", + "note": "Good for us to know, but we're sometimes okay with localized complexity when we have good test coverage." }, "duplicate_key": { "level": "error" }, "empty_constructor_needs_parens": { - "level": "ignore" + "level": "ignore", + "note": "We consider this idiomatic: `new NotFoundError`" }, "ensure_comprehensions": { - "level": "warn" + "level": "warn", + "note": "TODO: What is this?" }, "indentation": { "value": 4, "level": "error" }, "line_endings": { - "level": "ignore", + "level": "error", "value": "unix" }, "max_line_length": { @@ -48,40 +51,46 @@ }, "missing_fat_arrows": { "level": "ignore", - "is_strict": false + "is_strict": false, + "note": "There are plenty of dynamic `this` cases beyond simple class methods: event handlers, dynamic properties, even Mocha tests." }, "newlines_after_classes": { - "value": 3, - "level": "ignore" + "value": 2, + "level": "warn", + "note": "We prefer this, but it won't be a show-stopper." }, "no_backticks": { "level": "error" }, "no_debugger": { - "level": "warn" + "level": "error" }, "no_empty_functions": { - "level": "ignore" + "level": "error" }, "no_empty_param_list": { - "level": "ignore" + "level": "error" }, "no_implicit_braces": { - "level": "ignore", - "strict": true + "level": "warn", + "strict": false, + "note": "TODO: This fails on multi-line cases sometimes. https://github.com/clutchski/coffeelint/issues/459" }, "no_implicit_parens": { "strict": true, - "level": "ignore" + "level": "ignore", + "note": "This is definitely idiomatic CoffeeScript: `foo a, b, c`" }, "no_interpolation_in_single_quotes": { - "level": "ignore" + "level": "error" }, "no_plusplus": { - "level": "ignore" + "level": "ignore", + "note": "We're okay with the use of ++ and --." }, "no_stand_alone_at": { - "level": "ignore" + "level": "ignore", + "note": "We're okay with standalone @." }, "no_tabs": { "level": "error" @@ -95,28 +104,34 @@ "no_trailing_whitespace": { "level": "error", "allowed_in_comments": false, - "allowed_in_empty_lines": true + "allowed_in_empty_lines": false }, "no_unnecessary_double_quotes": { - "level": "ignore" + "level": "warn", + "note": "We prefer single quotes wherever possible, but this won't be a show-stopper." }, "no_unnecessary_fat_arrows": { - "level": "warn" + "level": "ignore", + "note": "We're okay with preemptive uses of fat arrows." }, "non_empty_constructor_needs_parens": { - "level": "ignore" + "level": "ignore", + "note": "Similar to `no_implicit_parens`, this is definitely idiomatic CoffeeScript: `new Foo a, b, c`" }, "prefer_english_operator": { - "level": "ignore", + "level": "error", "doubleNotLevel": "ignore" }, "space_operators": { - "level": "ignore" + "level": "ignore", + "note": "TODO: We're okay with no spaces for default params (e.g. `opts={}`). Exception being worked on: https://github.com/clutchski/coffeelint/pull/438" }, "spacing_after_comma": { - "level": "ignore" + "level": "ignore", + "note": "This is definitely idiomatic CoffeeScript: `foo a, b, c`" }, "transform_messes_up_line_numbers": { - "level": "warn" + "level": "warn", + "note": "This is just a recommended setting for CoffeeLint." } } diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 0fe6ecb..4cbac2f 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -149,7 +149,11 @@ module.exports = class GraphDatabase ## CYPHER + # NOTE: This method is fairly complex, in part out of necessity. + # We're okay with it since we test it throughly and emphasize its coverage. + # coffeelint: disable=cyclomatic_complexity cypher: (opts={}, cb, _tx) -> + # coffeelint: enable=cyclomatic_complexity if typeof opts is 'string' opts = {query: opts} @@ -200,7 +204,7 @@ module.exports = class GraphDatabase path = '/db/data/transaction' path += "/#{_tx._id}" if _tx?._id - path += "/commit" if commit or not _tx + path += '/commit' if commit or not _tx # Normalize input query or queries to an array of queries always, # but remember whether a single query was given (not a batch). @@ -233,9 +237,13 @@ module.exports = class GraphDatabase # NOTE: Lowercase 'rest' matters here for parsing. formats.push format = if lean then 'row' else 'rest' - statement: query - parameters: params or {} - resultDataContents: [format] + # NOTE: Braces needed by CoffeeLint for now. + # https://github.com/clutchski/coffeelint/issues/459 + { + statement: query + parameters: params or {} + resultDataContents: [format] + } # TODO: Support streaming! # diff --git a/lib-new/Transaction.coffee b/lib-new/Transaction.coffee index 18092d5..8e2a749 100644 --- a/lib-new/Transaction.coffee +++ b/lib-new/Transaction.coffee @@ -66,7 +66,12 @@ module.exports = class Transaction when @expiresIn <= 0 then 'expired' else 'open' + # NOTE: CoffeeLint currently false positives on this next line. + # https://github.com/clutchski/coffeelint/issues/458 + # coffeelint: disable=no_implicit_braces cypher: (opts={}, cb) -> + # coffeelint: enable=no_implicit_braces + # Check predictable error cases to provide better messaging sooner. # All of these are `ClientErrors` within the `Transaction` category. # http://neo4j.com/docs/stable/status-codes.html#_status_codes diff --git a/test-new/constraints._coffee b/test-new/constraints._coffee index 529945a..90c1da2 100644 --- a/test-new/constraints._coffee +++ b/test-new/constraints._coffee @@ -220,36 +220,39 @@ describe 'Constraints', -> describe '(misc)', -> - it 'should require both label and property to query specific constraint', -> + fail = -> throw new Error 'Callback should not have been called' + + it 'should require both label and property + to query specific constraint', -> for fn in [ - -> DB.hasConstraint null, -> - -> DB.hasConstraint '', -> - -> DB.hasConstraint {}, -> - -> DB.hasConstraint TEST_LABEL, -> - -> DB.hasConstraint {label: TEST_LABEL}, -> - -> DB.hasConstraint {property: TEST_PROP}, -> + -> DB.hasConstraint null, fail + -> DB.hasConstraint '', fail + -> DB.hasConstraint {}, fail + -> DB.hasConstraint TEST_LABEL, fail + -> DB.hasConstraint {label: TEST_LABEL}, fail + -> DB.hasConstraint {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i it 'should require both label and property to create constraint', -> for fn in [ - -> DB.createConstraint null, -> - -> DB.createConstraint '', -> - -> DB.createConstraint {}, -> - -> DB.createConstraint TEST_LABEL, -> - -> DB.createConstraint {label: TEST_LABEL}, -> - -> DB.createConstraint {property: TEST_PROP}, -> + -> DB.createConstraint null, fail + -> DB.createConstraint '', fail + -> DB.createConstraint {}, fail + -> DB.createConstraint TEST_LABEL, fail + -> DB.createConstraint {label: TEST_LABEL}, fail + -> DB.createConstraint {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i it 'should require both label and property to drop constraint', -> for fn in [ - -> DB.dropConstraint null, -> - -> DB.dropConstraint '', -> - -> DB.dropConstraint {}, -> - -> DB.dropConstraint TEST_LABEL, -> - -> DB.dropConstraint {label: TEST_LABEL}, -> - -> DB.dropConstraint {property: TEST_PROP}, -> + -> DB.dropConstraint null, fail + -> DB.dropConstraint '', fail + -> DB.dropConstraint {}, fail + -> DB.dropConstraint TEST_LABEL, fail + -> DB.dropConstraint {label: TEST_LABEL}, fail + -> DB.dropConstraint {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i diff --git a/test-new/constructor._coffee b/test-new/constructor._coffee index 7c3cb5f..b36d29f 100644 --- a/test-new/constructor._coffee +++ b/test-new/constructor._coffee @@ -130,7 +130,7 @@ describe 'GraphDatabase::constructor', -> expectAuth db, USERNAME, PASSWORD it 'should support clearing auth via empty string option', -> - host = "auth.test:9876" + host = 'auth.test:9876' url = "https://#{USERNAME}:#{PASSWORD}@#{host}" db = new GraphDatabase @@ -142,7 +142,7 @@ describe 'GraphDatabase::constructor', -> expectNoAuth db it 'should support clearing auth via empty object option', -> - host = "auth.test:9876" + host = 'auth.test:9876' url = "https://#{USERNAME}:#{PASSWORD}@#{host}" db = new GraphDatabase diff --git a/test-new/cypher._coffee b/test-new/cypher._coffee index fccf5fd..45feacc 100644 --- a/test-new/cypher._coffee +++ b/test-new/cypher._coffee @@ -83,8 +83,9 @@ describe 'GraphDatabase::cypher', -> expect(results).to.be.empty() it 'should reject empty/missing queries', -> - fn1 = -> DB.cypher '', -> - fn2 = -> DB.cypher {}, -> + fail = -> throw new Error 'Callback should not have been called' + fn1 = -> DB.cypher '', fail + fn2 = -> DB.cypher {}, fail expect(fn1).to.throw TypeError, /query/i expect(fn2).to.throw TypeError, /query/i @@ -130,7 +131,7 @@ describe 'GraphDatabase::cypher', -> # results, no longer the deterministic order of [a, b, r]. # We overcome this by explicitly indexing and ordering. results = DB.cypher - query: """ + query: ''' START a = node({idA}) MATCH (a) -[r]-> (b) WITH [ @@ -143,7 +144,7 @@ describe 'GraphDatabase::cypher', -> inner: obj.elmt }] AS outer ORDER BY i - """ + ''' params: idA: TEST_NODE_A._id , _ @@ -172,11 +173,11 @@ describe 'GraphDatabase::cypher', -> it 'should not parse nodes & relationships if lean', (_) -> results = DB.cypher - query: """ + query: ''' START a = node({idA}) MATCH (a) -[r]-> (b) RETURN a, b, r - """ + ''' params: idA: TEST_NODE_A._id lean: true diff --git a/test-new/fixtures/index._coffee b/test-new/fixtures/index._coffee index 9fb21e8..8f78bce 100644 --- a/test-new/fixtures/index._coffee +++ b/test-new/fixtures/index._coffee @@ -38,8 +38,8 @@ neo4j = require '../../' @DB_VERSION_NUM = parseFloat @DB_VERSION_STR, 10 if @DB_VERSION_NUM < 2 - throw new Error "*** node-neo4j v2 supports Neo4j v2+ only, - and you’re running Neo4j v1. These tests will fail! ***" + throw new Error '*** node-neo4j v2 supports Neo4j v2+ only, + and you’re running Neo4j v1. These tests will fail! ***' # # Creates and returns a property bag (dictionary) with unique, random test data diff --git a/test-new/http._coffee b/test-new/http._coffee index cdc3b46..0d68779 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -288,11 +288,11 @@ describe 'GraphDatabase::http', -> method: 'POST' path: '/db/data/cypher' body: - query: """ + query: ''' START a = node({idA}) MATCH (a) -[r]-> (b) RETURN a, r, b - """ + ''' params: idA: TEST_NODE_A._id , _ diff --git a/test-new/indexes._coffee b/test-new/indexes._coffee index b17a6c5..6844465 100644 --- a/test-new/indexes._coffee +++ b/test-new/indexes._coffee @@ -223,36 +223,38 @@ describe 'Indexes', -> describe '(misc)', -> + fail = -> throw new Error 'Callback should not have been called' + it 'should require both label and property to query specific index', -> for fn in [ - -> DB.hasIndex null, -> - -> DB.hasIndex '', -> - -> DB.hasIndex {}, -> - -> DB.hasIndex TEST_LABEL, -> - -> DB.hasIndex {label: TEST_LABEL}, -> - -> DB.hasIndex {property: TEST_PROP}, -> + -> DB.hasIndex null, fail + -> DB.hasIndex '', fail + -> DB.hasIndex {}, fail + -> DB.hasIndex TEST_LABEL, fail + -> DB.hasIndex {label: TEST_LABEL}, fail + -> DB.hasIndex {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i it 'should require both label and property to create index', -> for fn in [ - -> DB.createIndex null, -> - -> DB.createIndex '', -> - -> DB.createIndex {}, -> - -> DB.createIndex TEST_LABEL, -> - -> DB.createIndex {label: TEST_LABEL}, -> - -> DB.createIndex {property: TEST_PROP}, -> + -> DB.createIndex null, fail + -> DB.createIndex '', fail + -> DB.createIndex {}, fail + -> DB.createIndex TEST_LABEL, fail + -> DB.createIndex {label: TEST_LABEL}, fail + -> DB.createIndex {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i it 'should require both label and property to drop index', -> for fn in [ - -> DB.dropIndex null, -> - -> DB.dropIndex '', -> - -> DB.dropIndex {}, -> - -> DB.dropIndex TEST_LABEL, -> - -> DB.dropIndex {label: TEST_LABEL}, -> - -> DB.dropIndex {property: TEST_PROP}, -> + -> DB.dropIndex null, fail + -> DB.dropIndex '', fail + -> DB.dropIndex {}, fail + -> DB.dropIndex TEST_LABEL, fail + -> DB.dropIndex {label: TEST_LABEL}, fail + -> DB.dropIndex {property: TEST_PROP}, fail ] expect(fn).to.throw TypeError, /label and property required/i diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 0f00076..69b0d0c 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -79,11 +79,19 @@ describe 'Transactions', -> WHERE #{( # NOTE: Cypher doesn’t support directly comparing nodes and # property bags, so we have to compare each property. + # HACK: CoffeeLint thinks the below is bad indentation. + # https://github.com/clutchski/coffeelint/issues/456 + # coffeelint: disable=indentation for prop of properties "node.#{prop} = {properties}.#{prop}" + # coffeelint: enable=indentation + # HACK: CoffeeLint also thinks the below is double quotes! + # https://github.com/clutchski/coffeelint/issues/368 + # coffeelint: disable=no_unnecessary_double_quotes ).join ' AND '} RETURN node """ + # coffeelint: enable=no_unnecessary_double_quotes params: {properties} , _ diff --git a/test-new/util/helpers._coffee b/test-new/util/helpers._coffee index 825c95a..65152d8 100644 --- a/test-new/util/helpers._coffee +++ b/test-new/util/helpers._coffee @@ -77,7 +77,7 @@ neo4j = require '../../' # # NOTE: This assumes this error is returned from an HTTP response. # -@expectOldError = (err, statusCode, shortName, longName, message) -> +@expectOldError = (err, statusCode, shortName, longName, message) => ErrorType = if statusCode >= 500 then 'Database' else 'Client' @_expectBaseError err, "#{ErrorType}Error" From ae855db3de69a42fb18053a4702403773be702dd Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 7 Jun 2015 21:58:37 -0400 Subject: [PATCH 072/121] v2 / Tests: update for Neo4j 2.2.2, which fixes bugs. --- test-new/constraints._coffee | 2 +- test-new/http._coffee | 27 ++++++++++++++------------- test-new/indexes._coffee | 2 +- 3 files changed, 16 insertions(+), 15 deletions(-) diff --git a/test-new/constraints._coffee b/test-new/constraints._coffee index 90c1da2..f5d58cf 100644 --- a/test-new/constraints._coffee +++ b/test-new/constraints._coffee @@ -169,7 +169,7 @@ describe 'Constraints', -> # case, but previous versions return an old-style error. try helpers.expectError err, 'ClientError', 'Schema', - 'ConstraintAlreadyExists', expMessage + 'ConstraintViolation', expMessage catch assertionErr # Check for the older case, but in case it fails, # throw the original assertion error, not a new one. diff --git a/test-new/http._coffee b/test-new/http._coffee index 0d68779..c6c1bda 100644 --- a/test-new/http._coffee +++ b/test-new/http._coffee @@ -125,21 +125,19 @@ describe 'GraphDatabase::http', -> expect(err).to.exist() expect(body).to.not.exist() - # TEMP: Neo4j 2.2 responds here with a new-style error object, - # but it's currently a `DatabaseError` in 2.2.0. - # https://github.com/neo4j/neo4j/issues/4145 + # Neo4j 2.2 returns a proper new-style error object for this case, + # but previous versions return an old-style error. try - helpers.expectOldError err, 404, 'NodeNotFoundException', - 'org.neo4j.server.rest.web.NodeNotFoundException', + helpers.expectError err, + 'ClientError', 'Statement', 'EntityNotFound', 'Cannot find node with id [-1] in database.' catch assertionErr - # Check for the Neo4j 2.2 case, but if this fails, + # Check for the older case, but if this fails, # throw the original assertion error, not this one. try - helpers.expectError err, - 'DatabaseError', 'General', 'UnknownFailure', - 'org.neo4j.server.rest.web.NodeNotFoundException: - Cannot find node with id [-1] in database.' + helpers.expectOldError err, 404, 'NodeNotFoundException', + 'org.neo4j.server.rest.web.NodeNotFoundException', + 'Cannot find node with id [-1] in database.' catch doubleErr throw assertionErr @@ -202,7 +200,8 @@ describe 'GraphDatabase::http', -> expect(body.exception).to.equal 'NodeNotFoundException' expect(body.message).to.equal ' Cannot find node with id [-1] in database.' - expect(body.stacktrace).to.be.an 'array' + # Neo4j 2.2 changed `stacktrace` to `stackTrace`: + expect(body.stackTrace or body.stacktrace).to.be.an 'array' done() @@ -224,7 +223,8 @@ describe 'GraphDatabase::http', -> expect(body.exception).to.equal 'PropertyValueException' expect(body.message).to.equal 'Could not set property "object", unsupported type: {foo={bar=baz}}' - expect(body.stacktrace).to.be.an 'array' + # Neo4j 2.2 changed `stacktrace` to `stackTrace`: + expect(body.stackTrace or body.stacktrace).to.be.an 'array' done() @@ -254,7 +254,8 @@ describe 'GraphDatabase::http', -> expect(body.exception).to.equal 'PropertyValueException' expect(body.message).to.equal 'Could not set property "object", unsupported type: {foo={bar=baz}}' - expect(body.stacktrace).to.be.an 'array' + # Neo4j 2.2 changed `stacktrace` to `stackTrace`: + expect(body.stackTrace or body.stacktrace).to.be.an 'array' done() diff --git a/test-new/indexes._coffee b/test-new/indexes._coffee index 6844465..b2cff8c 100644 --- a/test-new/indexes._coffee +++ b/test-new/indexes._coffee @@ -172,7 +172,7 @@ describe 'Indexes', -> # case, but previous versions return an old-style error. try helpers.expectError err, 'ClientError', 'Schema', - 'IndexAlreadyExists', expMessage + 'ConstraintViolation', expMessage catch assertionErr # Check for the older case, but in case it fails, # throw the original assertion error, not a new one. From be547029d87295bb44af6d1716b238b22b088f38 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 7 Jun 2015 21:58:52 -0400 Subject: [PATCH 073/121] v2 / Tests: test Neo4j 2.3.0-M01 in CI. --- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index cf5f334..6eb0297 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,7 +10,8 @@ jdk: env: # test across multiple versions of Neo4j: - - NEO4J_VERSION="2.2.0" + - NEO4J_VERSION="2.3.0-M01" + - NEO4J_VERSION="2.2.2" - NEO4J_VERSION="2.1.7" - NEO4J_VERSION="2.0.4" From 5c16aeb83efda16c5b09411d252c5b6e197f0735 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 9 Jun 2015 20:52:57 -0400 Subject: [PATCH 074/121] v2 / Transactions: add proper state enum/constants. --- API_v2.md | 3 +- lib-new/Transaction.coffee | 111 ++++++++++++++++++---------------- test-new/transactions._coffee | 62 +++++++++---------- 3 files changed, 91 insertions(+), 85 deletions(-) diff --git a/API_v2.md b/API_v2.md index ff65ab1..5d9666e 100644 --- a/API_v2.md +++ b/API_v2.md @@ -254,7 +254,8 @@ which may be signs of code bugs, and provide more helpful error messaging. ```coffee class Transaction {_id, expiresAt, expiresIn, state} # `expiresAt` is a Date, while `expiresIn` is a millisecond count. -# `state` is one of 'open', 'pending', 'committed, 'rolled back', or 'expired'. +# `state` is one of 'open', 'pending', 'committed, 'rolled back', or 'expired', +# all of which are constant properties on instances, e.g. `STATE_ROLLED_BACK`. ``` ```js diff --git a/lib-new/Transaction.coffee b/lib-new/Transaction.coffee index 8e2a749..bcab330 100644 --- a/lib-new/Transaction.coffee +++ b/lib-new/Transaction.coffee @@ -16,55 +16,60 @@ module.exports = class Transaction @_expires = null @_pending = false @_committed = false - @_rolledback = false - - Object.defineProperty @::, 'expiresAt', - enumerable: true - get: -> - if @_expires - new Date @_expires - else - # This transaction hasn't been created yet, so far future: - new Date FAR_FUTURE_MS - - Object.defineProperty @::, 'expiresIn', - enumerable: true - get: -> - if @_expires - @expiresAt - (new Date) - else - # This transaction hasn't been created yet, so far future. - # Unlike for the Date instance above, we can be less arbitrary; - # hopefully it's never a problem to return Infinity here. - Infinity + @_rolledBack = false + + # Convenience helper for nice getter syntax: + # (NOTE: This does *not* support sibling setters, as we don't need it, + # but it's possible to add support.) + get = (props) => + for name, getter of props + Object.defineProperty @::, name, + configurable: true # For developer-friendliness, e.g. tweaking. + enumerable: true # So these show up in console.log, etc. + get: getter + + # This nice getter syntax uses implicit braces, however. + # coffeelint: disable=no_implicit_braces - # - # The state of this transaction. Returns one of the following values: - # - # - open - # - pending (a request is in progress) - # - committed - # - rolled back - # - expired - # - # TODO: Should we make this an enum? Or constants? - # - Object.defineProperty @::, 'state', - get: -> switch - # Order matters here. - # - # E.g. a request could have been made just before the expiry time, - # and we won't know the new expiry time until the server responds. - # - # TODO: The server could also receive it just *after* the expiry - # time, which'll cause it to return an unhelpful `UnknownId` error; - # should we handle that edge case in our `cypher` callback below? - # - when @_pending then 'pending' - when @_committed then 'committed' - when @_rolledback then 'rolled back' - when @expiresIn <= 0 then 'expired' - else 'open' + get STATE_OPEN: -> 'open' + get STATE_PENDING: -> 'pending' + get STATE_COMMITTED: -> 'committed' + get STATE_ROLLED_BACK: -> 'rolled back' + get STATE_EXPIRED: -> 'expired' + + get expiresAt: -> + if @_expires + new Date @_expires + else + # This transaction hasn't been created yet, so far future: + new Date FAR_FUTURE_MS + + get expiresIn: -> + if @_expires + @expiresAt - (new Date) + else + # This transaction hasn't been created yet, so far future. + # Unlike for the Date instance above, we can be less arbitrary; + # hopefully it's never a problem to return Infinity here. + Infinity + + get state: -> switch + # Order matters here. + # E.g. a request could have been made just before the expiry time, + # and we won't know the new expiry time until the server responds. + # + # TODO: The server could also receive it just *after* the expiry + # time, which'll cause it to return an unhelpful `UnknownId` error; + # should we handle that edge case in our `cypher` callback below? + # + when @_pending then @STATE_PENDING + when @_committed then @STATE_COMMITTED + when @_rolledBack then @STATE_ROLLED_BACK + when @expiresIn <= 0 then @STATE_EXPIRED + else @STATE_OPEN + + # For the above getters. + # coffeelint: enable=no_implicit_braces # NOTE: CoffeeLint currently false positives on this next line. # https://github.com/clutchski/coffeelint/issues/458 @@ -76,22 +81,22 @@ module.exports = class Transaction # All of these are `ClientErrors` within the `Transaction` category. # http://neo4j.com/docs/stable/status-codes.html#_status_codes errMsg = switch @state - when 'pending' + when @STATE_PENDING # This would otherwise throw a `ConcurrentRequest` error. 'A request within this transaction is currently in progress. Concurrent requests within a transaction are not allowed.' - when 'expired' + when @STATE_EXPIRED # This would otherwise throw an `UnknownId` error. 'This transaction has expired. You can get the expiration time of a transaction through its `expiresAt` (Date) and `expiresIn` (ms) properties. To prevent a transaction from expiring, execute any action or call `renew` before the transaction expires.' - when 'committed' + when @STATE_COMMITTED # This would otherwise throw an `UnknownId` error. 'This transaction has been committed. Transactions cannot be reused; begin a new one instead.' - when 'rolled back' + when @STATE_ROLLED_BACK # This would otherwise throw an `UnknownId` error. 'This transaction has been rolled back. Transactions get automatically rolled back on any @@ -123,7 +128,7 @@ module.exports = class Transaction if opts.commit and not err @_committed = true else - @_rolledback = true + @_rolledBack = true cb err, results , @ diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 69b0d0c..ae88eaa 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -29,15 +29,15 @@ describe 'Transactions', -> it 'should convey pending state, and reject concurrent requests', (done) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN fn = -> tx.cypher 'RETURN "bar" AS foo', cb - expect(tx.state).to.equal 'pending' + expect(tx.state).to.equal tx.STATE_PENDING cb = (err, results) -> expect(err).to.not.exist() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN done() fn() @@ -112,9 +112,9 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'committing' - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.commit _ - expect(tx.state).to.equal 'committed' + expect(tx.state).to.equal tx.STATE_COMMITTED expect(-> tx.cypher 'RETURN "bar" AS foo') .to.throw neo4j.ClientError, /been committed/i @@ -133,10 +133,10 @@ describe 'Transactions', -> it 'should support committing before any queries', (_) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.commit _ - expect(tx.state).to.equal 'committed' + expect(tx.state).to.equal tx.STATE_COMMITTED it 'should support auto-committing', (_) -> tx = DB.beginTransaction() @@ -158,7 +158,7 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'auto-committing' expect(nodeA.properties.i).to.equal 1 - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN [{nodeA}] = tx.cypher query: ''' @@ -174,7 +174,7 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'auto-committing' expect(nodeA.properties.i).to.equal 2 - expect(tx.state).to.equal 'committed' + expect(tx.state).to.equal tx.STATE_COMMITTED expect(-> tx.cypher 'RETURN "bar" AS foo') .to.throw neo4j.ClientError, /been committed/i @@ -207,9 +207,9 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'rolling back' - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.rollback _ - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK expect(-> tx.cypher 'RETURN "bar" AS foo') .to.throw neo4j.ClientError, /been rolled back/i @@ -228,10 +228,10 @@ describe 'Transactions', -> it 'should support rolling back before any queries', (_) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.rollback _ - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK # NOTE: Skipping this test by default, because it's slow (we have to pause # one second; see note within) and not really a mission-critical feature. @@ -268,9 +268,9 @@ describe 'Transactions', -> oldExpiresAt = tx.expiresAt setTimeout _, 1000 # TODO: Provide visual feedback? - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.renew _ - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN expect(tx.expiresAt).to.be.an.instanceOf Date expect(tx.expiresAt).to.be.greaterThan new Date @@ -283,7 +283,7 @@ describe 'Transactions', -> # to commit or expire (since it touches the existing graph, and our last # step is to delete the existing graph), roll this transaction back. tx.rollback _ - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK # We also ensure that renewing didn't cause the transaction to commit. [{nodeA}] = DB.cypher @@ -332,7 +332,7 @@ describe 'Transactions', -> 'ParameterMissing', 'Expected a parameter named foo' cont() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # Because of that, the first query's effects should still be visible # within the transaction: @@ -352,9 +352,9 @@ describe 'Transactions', -> # NOTE: But the transaction won't commit successfully apparently, both # manually or automatically. So we manually rollback instead. # TODO: Is this a bug in Neo4j? Or my understanding? - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.rollback _ - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK # TODO: Similar to the note above this, is this right? Or is this either a # bug in Neo4j or my understanding? Should client errors never be fatal? @@ -396,7 +396,7 @@ describe 'Transactions', -> 'ParameterMissing', 'Expected a parameter named foo' cont() - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK expect(-> tx.cypher 'RETURN "bar" AS foo') .to.throw neo4j.ClientError, /been rolled back/i @@ -448,7 +448,7 @@ describe 'Transactions', -> 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK expect(-> tx.cypher 'RETURN "bar" AS foo') .to.throw neo4j.ClientError, /been rolled back/i @@ -469,7 +469,7 @@ describe 'Transactions', -> it 'should properly handle non-fatal errors on the first query', (_) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # For precision, implementing this step without Streamline. do (cont=_) => @@ -479,12 +479,12 @@ describe 'Transactions', -> 'ParameterMissing', 'Expected a parameter named foo' cont() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN it 'should properly handle fatal client errors on an auto-commit first query', (_) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # For precision, implementing this step without Streamline. do (cont=_) => @@ -497,11 +497,11 @@ describe 'Transactions', -> 'ParameterMissing', 'Expected a parameter named foo' cont() - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK it 'should properly handle fatal database errors on the first query', (_) -> tx = DB.beginTransaction() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # HACK: Depending on a known bug to trigger a DatabaseError; # that makes this test brittle, since the bug could get fixed! @@ -519,7 +519,7 @@ describe 'Transactions', -> 'scala.MatchError: (foo,null) (of class scala.Tuple2)' cont() - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK it 'should properly handle errors with batching', (_) -> tx = DB.beginTransaction() @@ -555,7 +555,7 @@ describe 'Transactions', -> expect(nodeA.properties.test).to.equal 'errors with batching' expect(nodeA.properties.i).to.equal 1 - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # Now trigger a client error within another batch; this should *not* # rollback (and thus destroy) the transaction. @@ -607,7 +607,7 @@ describe 'Transactions', -> cont() - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN # Because of that, the effects of the first query in the batch (before # the error) should still be visible within the transaction: @@ -631,9 +631,9 @@ describe 'Transactions', -> # NOTE: But the transaction won't commit successfully apparently, both # manually or automatically. So we manually rollback instead. # TODO: Is this a bug in Neo4j? Or my understanding? - expect(tx.state).to.equal 'open' + expect(tx.state).to.equal tx.STATE_OPEN tx.rollback _ - expect(tx.state).to.equal 'rolled back' + expect(tx.state).to.equal tx.STATE_ROLLED_BACK it 'should support streaming (TODO)' From a3b5dfc87d7112a0e8284f8d66327f9c86a85f80 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 10 Jun 2015 12:56:54 -0400 Subject: [PATCH 075/121] v2 / Schema: improve index & constraint idempotence. --- API_v2.md | 8 +++- lib-new/Constraint.coffee | 8 +++- lib-new/GraphDatabase.coffee | 84 +++++++++++++++++++++++++++++------- lib-new/Index.coffee | 8 +++- test-new/constraints._coffee | 55 ++++++++--------------- test-new/indexes._coffee | 55 ++++++++--------------- 6 files changed, 125 insertions(+), 93 deletions(-) diff --git a/API_v2.md b/API_v2.md index 5d9666e..3d413a6 100644 --- a/API_v2.md +++ b/API_v2.md @@ -411,7 +411,10 @@ db.getIndexes(cbMany); // across all labels db.getIndexes({label}, cbMany); // for a particular label db.hasIndex({label, property}, cbBool); db.createIndex({label, property}, cbOne); -db.dropIndex({label, property}, cbDone); + // callback receives null if this index already exists +db.dropIndex({label, property}, cbBool); + // callback receives true if this index existed and was dropped, + // false if it didn't exist in the first place ``` Returned indexes are minimal `Index` objects: @@ -446,7 +449,10 @@ db.getConstraints(cbMany); // across all labels db.getConstraints({label}, cbMany); // for a particular label db.hasConstraint({label, property}, cbBool); db.createConstraint({label, property}, cbOne); + // callback receives null if this constraint already exists db.dropConstraint({label, property}, cbDone); + // callback receives true if this constraint existed and was dropped, + // false if it didn't exist in the first place ``` Returned constraints are minimal `Constraint` objects: diff --git a/lib-new/Constraint.coffee b/lib-new/Constraint.coffee index a31dbe5..214bacc 100644 --- a/lib-new/Constraint.coffee +++ b/lib-new/Constraint.coffee @@ -28,8 +28,12 @@ module.exports = class Constraint node-neo4j v#{lib.version} doesn’t know how to handle these. Continuing as if it’s a UNIQUENESS constraint..." - # TODO: Neo4j always returns an array of property keys, but only one - # property key is supported today. Do we need to support multiple? + if property_keys.length > 1 + console.warn "Constraint (on :#{label}) with #{property_keys.length} + property keys encountered: #{property_keys.join ', '}. + node-neo4j v#{lib.version} doesn’t know how to handle these. + Continuing with only the first one." + [property] = property_keys return new Constraint {label, property} diff --git a/lib-new/GraphDatabase.coffee b/lib-new/GraphDatabase.coffee index 4cbac2f..0a9b05b 100644 --- a/lib-new/GraphDatabase.coffee +++ b/lib-new/GraphDatabase.coffee @@ -343,7 +343,6 @@ module.exports = class GraphDatabase # This endpoint returns the array of labels directly: # http://neo4j.com/docs/stable/rest-api-node-labels.html#rest-api-list-all-labels # Hence passing the callback directly. `http` handles 4xx, 5xx errors. - # TODO: Would it be better for us to handle other non-200 responses too? @http method: 'GET' path: '/db/data/labels' @@ -353,7 +352,6 @@ module.exports = class GraphDatabase # This endpoint returns the array of property keys directly: # http://neo4j.com/docs/stable/rest-api-property-values.html#rest-api-list-all-property-keys # Hence passing the callback directly. `http` handles 4xx, 5xx errors. - # TODO: Would it be better for us to handle other non-200 responses too? @http method: 'GET' path: '/db/data/propertykeys' @@ -363,7 +361,6 @@ module.exports = class GraphDatabase # This endpoint returns the array of relationship types directly: # http://neo4j.com/docs/stable/rest-api-relationship-types.html#rest-api-get-relationship-types # Hence passing the callback directly. `http` handles 4xx, 5xx errors. - # TODO: Would it be better for us to handle other non-200 responses too? @http method: 'GET' path: '/db/data/relationship/types' @@ -403,12 +400,10 @@ module.exports = class GraphDatabase # NOTE: This is just a convenience method; there is no REST API endpoint # for this directly (surprisingly, since there is for constraints). + # https://github.com/neo4j/neo4j/issues/4214 @getIndexes {label}, (err, indexes) -> - if err - cb err - else - cb null, indexes.some (index) -> - index.label is label and index.property is property + cb err, indexes?.some (index) -> + index.label is label and index.property is property createIndex: (opts={}, cb) -> {label, property} = opts @@ -417,12 +412,26 @@ module.exports = class GraphDatabase throw new TypeError \ 'Label and property required to create an index.' + # Passing `raw: true` so we can handle the 409 case below. @http method: 'POST' path: "/db/data/schema/index/#{encodeURIComponent label}" body: {'property_keys': [property]} - , (err, index) -> - cb err, if index then Index._fromRaw index + raw: true + , (err, resp) -> + if err + return cb err + + # Neo4j returns a 409 error (w/ varying code across versions) + # if this index already exists (including for a constraint). + if resp.statusCode is 409 + return cb null, null + + # Translate all other error responses as legitimate errors: + if err = Error._fromResponse resp + return cb err + + cb err, if resp.body then Index._fromRaw resp.body dropIndex: (opts={}, cb) -> {label, property} = opts @@ -432,12 +441,26 @@ module.exports = class GraphDatabase # This endpoint is void, i.e. returns nothing: # http://neo4j.com/docs/stable/rest-api-schema-indexes.html#rest-api-drop-index - # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + # Passing `raw: true` so we can handle the 409 case below. @http method: 'DELETE' path: "/db/data/schema/index\ /#{encodeURIComponent label}/#{encodeURIComponent property}" - , cb + raw: true + , (err, resp) -> + if err + return cb err + + # Neo4j returns a 404 response (with an empty body) + # if this index doesn't exist (has already been dropped). + if resp.statusCode is 404 + return cb null, false + + # Translate all other error responses as legitimate errors: + if err = Error._fromResponse resp + return cb err + + cb err, true # Index existed and was dropped ## CONSTRAINTS @@ -483,6 +506,7 @@ module.exports = class GraphDatabase # NOTE: A REST API endpoint *does* exist to get a specific constraint: # http://neo4j.com/docs/stable/rest-api-schema-constraints.html # But it (a) returns an array, and (b) throws a 404 if no constraint. + # https://github.com/neo4j/neo4j/issues/4214 # For those reasons, it's actually easier to just fetch all constraints; # no error handling needed, and array processing either way. # @@ -514,13 +538,27 @@ module.exports = class GraphDatabase # NOTE: We explicitly *are* assuming uniqueness type here, since # that's our only option today for creating constraints. + # NOTE: Passing `raw: true` so we can handle the 409 case below. @http method: 'POST' path: "/db/data/schema/constraint\ /#{encodeURIComponent label}/uniqueness" body: {'property_keys': [property]} - , (err, constraint) -> - cb err, if constraint then Constraint._fromRaw constraint + raw: true + , (err, resp) -> + if err + return cb err + + # Neo4j returns a 409 error (w/ varying code across versions) + # if this constraint already exists. + if resp.statusCode is 409 + return cb null, null + + # Translate all other error responses as legitimate errors: + if err = Error._fromResponse resp + return cb err + + cb err, if resp.body then Constraint._fromRaw resp.body dropConstraint: (opts={}, cb) -> # TODO: We may need to support an additional `type` param too, @@ -533,12 +571,26 @@ module.exports = class GraphDatabase # This endpoint is void, i.e. returns nothing: # http://neo4j.com/docs/stable/rest-api-schema-constraints.html#rest-api-drop-constraint - # Hence passing the callback directly. `http` handles 4xx, 5xx errors. + # Passing `raw: true` so we can handle the 409 case below. @http method: 'DELETE' path: "/db/data/schema/constraint/#{encodeURIComponent label}\ /uniqueness/#{encodeURIComponent property}" - , cb + raw: true + , (err, resp) -> + if err + return cb err + + # Neo4j returns a 404 response (with an empty body) + # if this constraint doesn't exist (has already been dropped). + if resp.statusCode is 404 + return cb null, false + + # Translate all other error responses as legitimate errors: + if err = Error._fromResponse resp + return cb err + + cb err, true # Constraint existed and was dropped # TODO: Legacy indexing diff --git a/lib-new/Index.coffee b/lib-new/Index.coffee index bb89c0b..d807093 100644 --- a/lib-new/Index.coffee +++ b/lib-new/Index.coffee @@ -18,8 +18,12 @@ module.exports = class Index @_fromRaw: (obj) -> {label, property_keys} = obj - # TODO: Neo4j always returns an array of property keys, but only one - # property key is supported today. Do we need to support multiple? + if property_keys.length > 1 + console.warn "Index (on :#{label}) with #{property_keys.length} + property keys encountered: #{property_keys.join ', '}. + node-neo4j v#{lib.version} doesn’t know how to handle these. + Continuing with only the first one." + [property] = property_keys return new Index {label, property} diff --git a/test-new/constraints._coffee b/test-new/constraints._coffee index f5d58cf..48c9ac1 100644 --- a/test-new/constraints._coffee +++ b/test-new/constraints._coffee @@ -103,12 +103,12 @@ describe 'Constraints', -> ORIG_CONSTRAINTS_LABEL = constraints it 'should support querying for specific constraint', (_) -> - bool = DB.hasConstraint + exists = DB.hasConstraint label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal false + expect(exists).to.equal false it '(verify constraint doesn’t exist yet)', (_) -> # This shouldn't throw an error: @@ -138,12 +138,12 @@ describe 'Constraints', -> expect(constraints).to.contain TEST_CONSTRAINT it '(verify by re-querying specific test constraint)', (_) -> - bool = DB.hasConstraint + exists = DB.hasConstraint label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal true + expect(exists).to.equal true it '(verify with test query)', (done) -> violateConstraint (err) -> @@ -157,39 +157,22 @@ describe 'Constraints', -> done() - it 'should throw on create of already-created constraint', (done) -> - DB.createConstraint + it 'should be idempotent on repeat creates of constraint', (_) -> + constraint = DB.createConstraint label: TEST_LABEL property: TEST_PROP - , (err, constraint) -> - expMessage = "Label '#{TEST_LABEL}' and property '#{TEST_PROP}' - already have a unique constraint defined on them." - - # Neo4j 2.2 returns a proper new-style error object for this - # case, but previous versions return an old-style error. - try - helpers.expectError err, 'ClientError', 'Schema', - 'ConstraintViolation', expMessage - catch assertionErr - # Check for the older case, but in case it fails, - # throw the original assertion error, not a new one. - try - helpers.expectOldError err, 409, - 'ConstraintViolationException', - 'org.neo4j.graphdb.ConstraintViolationException', - expMessage - catch doubleErr - throw assertionErr - - expect(constraint).to.not.exist() - done() + , _ + + expect(constraint).to.not.exist() it 'should support dropping constraint', (_) -> - DB.dropConstraint + dropped = DB.dropConstraint label: TEST_LABEL property: TEST_PROP , _ + expect(dropped).to.equal true + describe '(after constraint dropped)', -> @@ -202,20 +185,20 @@ describe 'Constraints', -> expect(constraints).to.eql ORIG_CONSTRAINTS_LABEL it '(verify by re-querying specific test constraint)', (_) -> - bool = DB.hasConstraint + exists = DB.hasConstraint label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal false + expect(exists).to.equal false - it 'should throw on drop of already-dropped constraint', (done) -> - DB.dropConstraint + it 'should be idempotent on repeat drops of constraint', (_) -> + dropped = DB.dropConstraint label: TEST_LABEL property: TEST_PROP - , (err) -> - helpers.expectHttpError err, 404 - done() + , _ + + expect(dropped).to.equal false describe '(misc)', -> diff --git a/test-new/indexes._coffee b/test-new/indexes._coffee index b2cff8c..00d2cb2 100644 --- a/test-new/indexes._coffee +++ b/test-new/indexes._coffee @@ -96,12 +96,12 @@ describe 'Indexes', -> ORIG_INDEXES_LABEL = indexes it 'should support querying for specific index', (_) -> - bool = DB.hasIndex + exists = DB.hasIndex label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal false + expect(exists).to.equal false it '(verify index doesn’t exist yet)', (done) -> DB.cypher TEST_CYPHER, (err, results) -> @@ -141,12 +141,12 @@ describe 'Indexes', -> expect(indexes).to.contain TEST_INDEX it '(verify by re-querying specific test index)', (_) -> - bool = DB.hasIndex + exists = DB.hasIndex label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal true + expect(exists).to.equal true # TODO: This sometimes fails because the index hasn't come online yet, # but Neo4j's REST API doesn't return index online/offline status. @@ -160,39 +160,22 @@ describe 'Indexes', -> n: TEST_NODE_B ] - it 'should throw on create of already-created index', (done) -> - DB.createIndex + it 'should be idempotent on repeat creates of index', (_) -> + index = DB.createIndex label: TEST_LABEL property: TEST_PROP - , (err, index) -> - expMessage = "There already exists an index - for label '#{TEST_LABEL}' on property '#{TEST_PROP}'." - - # Neo4j 2.2 returns a proper new-style error object for this - # case, but previous versions return an old-style error. - try - helpers.expectError err, 'ClientError', 'Schema', - 'ConstraintViolation', expMessage - catch assertionErr - # Check for the older case, but in case it fails, - # throw the original assertion error, not a new one. - try - helpers.expectOldError err, 409, - 'ConstraintViolationException', - 'org.neo4j.graphdb.ConstraintViolationException', - expMessage - catch doubleErr - throw assertionErr - - expect(index).to.not.exist() - done() + , _ + + expect(index).to.not.exist() it 'should support dropping index', (_) -> - DB.dropIndex + dropped = DB.dropIndex label: TEST_LABEL property: TEST_PROP , _ + expect(dropped).to.equal true + describe '(after index dropped)', -> @@ -205,20 +188,20 @@ describe 'Indexes', -> expect(indexes).to.eql ORIG_INDEXES_LABEL it '(verify by re-querying specific test index)', (_) -> - bool = DB.hasIndex + exists = DB.hasIndex label: TEST_LABEL property: TEST_PROP , _ - expect(bool).to.equal false + expect(exists).to.equal false - it 'should throw on drop of already-dropped index', (done) -> - DB.dropIndex + it 'should be idempotent on repeat drops of constraint', (_) -> + dropped = DB.dropIndex label: TEST_LABEL property: TEST_PROP - , (err) -> - helpers.expectHttpError err, 404 - done() + , _ + + expect(dropped).to.equal false describe '(misc)', -> From 8896cf09c8ffd30ecff2166e73c381fb4735fea7 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 11 Jun 2015 09:30:11 -0400 Subject: [PATCH 076/121] v2 / Misc: update version to 2.0.0-RC2. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index cc12d22..32baa3a 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "neo4j", "description": "Neo4j driver (REST API client) for Node.js", - "version": "2.0.0-RC1", + "version": "2.0.0-RC2", "author": "Aseem Kishore ", "contributors": [ "Daniel Gasienica ", From 3feb20572a4ae4cd33825c0c5e07816ff3e8620e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 17 Jun 2015 10:45:21 -0400 Subject: [PATCH 077/121] v2 / Docs: move dev info to new CONTRIBUTING.md. [skip ci] --- CONTRIBUTING.md | 39 +++++++++++++++++++++++++++++++++++++++ README.md | 41 ----------------------------------------- 2 files changed, 39 insertions(+), 41 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..5e939cf --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,39 @@ +## Development + + git clone git@github.com:thingdom/node-neo4j.git + cd node-neo4j + npm install && npm run clean + +You'll need a local installation of Neo4j ([links](http://neo4j.org/download)), +and it should be running on the default port of 7474 (`neo4j start`). + +To run the tests: + + npm test + +This library is written in [CoffeeScript][], using [Streamline.js][] syntax. +The tests automatically compile the code on-the-fly, but you can also generate +compiled `.js` files from the source `._coffee` files manually: + + npm run build + +This is in fact what's run each time this library is published to npm. +But please don't check the generated `.js` files in; to remove: + + npm run clean + +When compiled `.js` files exist, changes to the source `._coffee` files will +*not* be picked up automatically; you'll need to rebuild. + +If you `npm link` this module into another app (like [node-neo4j-template][]) +and you want the code compiled on-the-fly during development, you can create +an `index.js` file under `lib/` with the following: + +```js +require('coffee-script/register'); +require('streamline/register'); +module.exports = require('./index._coffee'); +``` + +But don't check this in! That would cause all clients to compile the code +on-the-fly every time, which isn't desirable in production. diff --git a/README.md b/README.md index d866c1f..f73da3a 100644 --- a/README.md +++ b/README.md @@ -140,47 +140,6 @@ your package.json dependency as e.g. `1.x` or `^1.0` instead of `*`. [Roadmap]: https://github.com/thingdom/node-neo4j/wiki/Roadmap -## Development - - git clone git@github.com:thingdom/node-neo4j.git - cd node-neo4j - npm install && npm run clean - -You'll need a local installation of Neo4j ([links](http://neo4j.org/download)), -and it should be running on the default port of 7474 (`neo4j start`). - -To run the tests: - - npm test - -This library is written in [CoffeeScript][], using [Streamline.js][] syntax. -The tests automatically compile the code on-the-fly, but you can also generate -compiled `.js` files from the source `._coffee` files manually: - - npm run build - -This is in fact what's run each time this library is published to npm. -But please don't check the generated `.js` files in; to remove: - - npm run clean - -When compiled `.js` files exist, changes to the source `._coffee` files will -*not* be picked up automatically; you'll need to rebuild. - -If you `npm link` this module into another app (like [node-neo4j-template][]) -and you want the code compiled on-the-fly during development, you can create -an `index.js` file under `lib/` with the following: - -```js -require('coffee-script/register'); -require('streamline/register'); -module.exports = require('./index._coffee'); -``` - -But don't check this in! That would cause all clients to compile the code -on-the-fly every time, which isn't desirable in production. - - ## Changes See the [Changelog][changelog] for the full history of changes and releases. From d6f55a5423c93201ae1de2b68fd0a69704de23e2 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 17 Jun 2015 19:16:35 -0400 Subject: [PATCH 078/121] v2 / Docs: flesh out CONTRIBUTING.md. --- CONTRIBUTING.md | 118 ++++++++++++++++++++++++++++++++++++--------- test-new/README.md | 33 ------------- 2 files changed, 96 insertions(+), 55 deletions(-) delete mode 100644 test-new/README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5e939cf..14697e4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,39 +1,113 @@ +## Issues + +Bug reports and feature requests are always welcome via [GitHub Issues](https://github.com/thingdom/node-neo4j/issues). General questions and troubleshooting are better served by the [mailing list](https://groups.google.com/group/node-neo4j) and [Stack Overflow](http://stackoverflow.com/questions/ask?tags=node.js,neo4j). + +For bug reports, please try to include: + +- Neo4j version (`curl -s localhost:7474/db/data/ | grep version`) +- Node.js version (`node --version`) +- node-neo4j version (`npm ls neo4j`) +- npm version (`npm --version`) +- Example code, if possible (pro tip: [create a gist](https://gist.github.com/) if there's a lot) + +For feature requests, real-world use cases are always helpful. + + +## Pull Requests + +If you're comfortable rolling up your sleeves and contributing code (instructions below), great! Fork this repo, start a new branch, and pull request away, but please follow these guidelines: + +- Follow the existing code style. (We use [CoffeeLint](http://coffeelint.org/) to check it.) `npm run lint` should continue to pass. + +- Add or update tests as appropriate. (You can look at the existing tests for reference.) `npm test` should continue to pass. + +- Add or update documentation similarly. But don't stress too much about it; we'll likely polish documentation ourselves before release anyway. + +- If this main repo's `master` branch has moved forward since you began, do merge the latest changes into your branch (or rebase your branch if you're comfortable with that). + +Thanks! We'll try to review your pull request as soon as we can. + + ## Development - git clone git@github.com:thingdom/node-neo4j.git - cd node-neo4j - npm install && npm run clean +To get set up for development, after cloning this repository: -You'll need a local installation of Neo4j ([links](http://neo4j.org/download)), -and it should be running on the default port of 7474 (`neo4j start`). +```sh +npm install && npm run clean +``` + +You'll need a [local installation of Neo4j](http://neo4j.org/download), and it should be running on the default port of 7474. To run the tests: - npm test +```sh +npm test +``` + +This library is written in [CoffeeScript](http://coffeescript.org/), and we lint the code with [CoffeeLint](http://coffeelint.org/). To lint: + +```sh +npm run lint +``` -This library is written in [CoffeeScript][], using [Streamline.js][] syntax. -The tests automatically compile the code on-the-fly, but you can also generate -compiled `.js` files from the source `._coffee` files manually: +The tests automatically compile the CoffeeScript on-the-fly, but you can also generate `.js` files from the source `.coffee` files manually: - npm run build +```sh +npm run build +``` -This is in fact what's run each time this library is published to npm. -But please don't check the generated `.js` files in; to remove: +This is in fact what's run on `prepublish` for npm. But please don't check the generated `.js` files in; to remove: - npm run clean +```sh +npm run clean +``` -When compiled `.js` files exist, changes to the source `._coffee` files will -*not* be picked up automatically; you'll need to rebuild. +When compiled `.js` files exist, changes to the source `.coffee` files will *not* be picked up automatically; you'll need to rebuild. -If you `npm link` this module into another app (like [node-neo4j-template][]) -and you want the code compiled on-the-fly during development, you can create -an `index.js` file under `lib/` with the following: +If you `npm link` this module and you want the code compiled on-the-fly during development, you can create an `exports.js` file under `lib-new/` with the following: ```js require('coffee-script/register'); -require('streamline/register'); -module.exports = require('./index._coffee'); +module.exports = require('./exports.coffee'); ``` -But don't check this in! That would cause all clients to compile the code -on-the-fly every time, which isn't desirable in production. +But don't check this in! That would cause all clients to compile the code on-the-fly every time, which isn't desirable in production. + + +## Testing + +This library strives for thorough test coverage. Every major feature has corresponding tests. + +The tests run on [Mocha](http://mochajs.org/) and use [Chai](http://chaijs.com/) for assertions. In addition, the code is written in [Streamline.js](https://github.com/Sage/streamlinejs) syntax, for convenience and robustness. + +In a nutshell, instead of calling async functions with a callback (which receives an error and a result), we get to call them with an `_` parameter instead — and then pretend as if the functions are synchronous (*returning* their results and *throwing* any errors). + +E.g. instead of writing tests like this: + +```coffee +it 'should get foo then set bar', (done) -> + db.getFoo (err, foo) -> + return done err if err + expect(foo).to.be.a 'number' + + db.setBar foo, (err, bar) -> + return done err if err + expect(bar).to.equal foo + + done() +``` + +We get to write tests like this: + +```coffee +it 'should get foo then set bar', (_) -> + foo = db.getFoo _ + expect(foo).to.be.a 'number' + + bar = db.setBar foo, _ + expect(bar).to.equal foo +``` + +This lets us write more concise tests that simultaneously test for errors more thoroughly. (If an async function "throws" an error, the test will fail.) + +It's important for our tests to pass across multiple versions of Neo4j, and they should also be robust to existing data (e.g. labels and constraints) in the database. To that end, they should generate data for testing, and then clean up that data at the end. diff --git a/test-new/README.md b/test-new/README.md deleted file mode 100644 index 28f5246..0000000 --- a/test-new/README.md +++ /dev/null @@ -1,33 +0,0 @@ -## Node-Neo4j Tests - -Many of these tests are written in [Streamline.js](https://github.com/Sage/streamlinejs) syntax, for convenience and robustness. - -In a nutshell, instead of calling async functions with a callback (that takes an error and a result), we get to call them with an `_` parameter instead, and then pretend as if the functions are synchronous (*returning* their results and *throwing* any errors). - -E.g. instead of writing tests like this: - -```coffee -it 'should get foo then set bar', (done) -> - db.getFoo (err, foo) -> - expect(err).to.not.exist() - expect(foo).to.be.a 'number' - - db.setBar foo, (err, bar) -> - expect(err).to.not.exist() - expect(bar).to.equal foo - - done() -``` - -We get to write tests like this: - -```coffee -it 'should get foo then set bar', (_) -> - foo = db.getFoo _ - expect(foo).to.be.a 'number' - - bar = db.setBar foo, _ - expect(bar).to.equal foo -``` - -This lets us write more concise tests that simultaneously test for errors more thoroughly. (If an async function "throws" an error, the test will fail.) From 612206804d461fe2b2e8a9979fc6fc931aa10a97 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Thu, 27 Aug 2015 11:05:05 -0700 Subject: [PATCH 079/121] v2 / Docs: overhaul readme; start manual. [skip ci] --- README.md | 268 +++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 194 insertions(+), 74 deletions(-) diff --git a/README.md b/README.md index f73da3a..091248c 100644 --- a/README.md +++ b/README.md @@ -1,67 +1,142 @@ - -# Node-Neo4j npm version +## Features -This is a [Node.js][node.js] driver for [Neo4j][neo4j] via it's [REST API][neo4j-rest-api]. +- **Cypher queries**, parameters, and **transactions** +- Arbitrary **HTTP requests**, for custom **Neo4j plugins** +- **Custom headers**, for **high availability**, application tracing, query logging, and more +- **Precise errors**, for robust error handling from the start +- Configurable **connection pooling**, for performance tuning & monitoring +- Thorough test coverage with **>100 tests** +- **Continuously integrated** against **multiple versions** of Node.js and Neo4j -**This driver has undergone a complete rewrite for Neo4j v2.** -It now *only* supports Neo4j 2.x — but it supports it really well. -(If you're still on Neo4j 1.x, you can still use -[node-neo4j v1](https://github.com/thingdom/node-neo4j/tree/v1).) -## What is Neo4j? +## Installation -Neo4j is a transactional, open-source graph database. A graph database manages data in a connected data structure, capable of representing any kind of data in a very accessible way. Information is stored in nodes and relationships connecting them, both of which can have arbitrary properties. To learn more visit [What is a Graph Database?][what-is-a-graph-database] +```sh +npm install neo4j --save +``` - +## Example - +```js +var neo4j = require('neo4j'); +var db = new neo4j.GraphDatabase('http://username:password@localhost:7474'); +db.cypher({ + query: 'MATCH (user:User {email: {email}}) RETURN user', + params: { + email: 'alice@example.com', + }, +}, callback); -## Installation +function callback(err, results) { + if (err) throw err; + var result = results[0]; + if (!result) { + console.log('No user found.'); + } else { + var user = result['user']; + console.log(JSON.stringify(user, null, 4)); + } +}; +``` -```sh -npm install neo4j --save +Yields e.g.: + +```json +{ + "_id": 12345678, + "labels": [ + "User", + "Admin" + ], + "properties": { + "name": "Alice Smith", + "email": "alice@example.com", + "emailVerified": true, + "passwordHash": "..." + } +} ``` -## Usage +See [node-neo4j-template](https://github.com/aseemk/node-neo4j-template) for a more thorough example. + +TODO: Also link to movies example. + + +## Basics + +Connect to a running Neo4j instance by instantiating the **`GraphDatabase`** class: ```js var neo4j = require('neo4j'); + +// Shorthand: var db = new neo4j.GraphDatabase('http://username:password@localhost:7474'); +// Full options: +var db = new neo4j.GraphDatabase({...}); +``` + +Options: + +- **`url` (required)**: the base URL to the Neo4j instance, e.g. `'http://localhost:7474'`. This can include auth credentials (e.g. `'http://username:password@localhost:7474'`), but doesn't have to. + +- **`auth`**: optional auth credentials; either a `'username:password'` string, or a `{username, password}` object. If present, this takes precedence over any credentials in the `url`. + +- **`headers`**: optional custom HTTP headers to send with every request. These can be overridden per request. Node-Neo4j defaults to sending a `User-Agent` identifying itself, but this can be overridden too. + +- **`proxy`**: optional URL to a proxy. If present, all requests will be routed through the proxy. + +- **`agent`**: optional [`http.Agent`](http://nodejs.org/api/http.html#http_http_agent) instance, for custom socket pooling. + +Once you have a `GraphDatabase` instance, you can make queries and more. + +Most operations are **asynchronous**, which means they take a **callback**. Node-Neo4j callbacks are of the standard `(error[, results])` form. + +Async control flow can get tricky quickly, so it's *highly* recommended to use a flow control library or tool, like [async](https://github.com/caolan/async) or [Streamline](https://github.com/Sage/streamlinejs). + + +## Cypher + +To make a Cypher query, simply pass the string query, any query parameters, and a callback to receive the error or results. + +```js db.cypher({ - query: 'MATCH (u:User {email: {email}}) RETURN u', + query: 'MATCH (user:User {email: {email}}) RETURN user', params: { email: 'alice@example.com', }, -}, function (err, results) { +}, callback); +``` + +It's extremely important to pass `params` separately. If you concatenate them into the `query`, you'll be vulnerable to injection attacks, and Neo4j performance will suffer as well. + +Cypher queries *always* return a list of results (like SQL rows), with each result having common properties (like SQL columns). Thus, query **results** passed to the callback are *always* an **array** (even if it's empty), and each **result** in the array is *always* an **object** (even if it's empty). + +```js +function callback(err, results) { if (err) throw err; var result = results[0]; if (!result) { console.log('No user found.'); } else { - var user = result['u']; + var user = result['user']; console.log(JSON.stringify(user, null, 4)); } -}); +}; ``` -Yields e.g.: +If the query results include nodes or relationships, **`Node`** and **`Relationship`** instances are returned for them. These instances encapsulate `{_id, labels, properties}` for nodes, and `{_id, type, properties, _fromId, _toId}` for relationships, but they can be used just like normal objects. ```json { @@ -79,65 +154,112 @@ Yields e.g.: } ``` -## Getting Help +(The `_id` properties refer to Neo4j's internal IDs. These can be convenient for debugging, but their use otherwise — especially externally — is discouraged.) -If you're having any issues you can first refer to the [API documentation][api-docs]. +If you don't need to know Neo4j IDs, node labels, or relationship types, you can pass `lean: true` to get back *just* properties, for a potential performance gain. -If you encounter any bugs or other issues, please file them in the -[issue tracker][issue-tracker]. +```js +db.cypher({ + query: 'MATCH (user:User {email: {email}}) RETURN user', + params: { + email: 'alice@example.com', + }, + lean: true, +}, callback); +``` -We also now have a [Google Group][google-group]! -Post questions and participate in general discussions there. +```json +{ + "name": "Alice Smith", + "email": "alice@example.com", + "emailVerified": true, + "passwordHash": "..." +} +``` -You can also [ask a question on StackOverflow][stackoverflow-ask] +Other options: +- **`headers`**: optional custom HTTP headers to send with this query. These will add onto the default `GraphDatabase` `headers`, but also override any that overlap. -## Neo4j version support -| **Version** | **Ver 1.x** | **Ver 2.x** | -|-------------|--------------|-------------| -| 1.5-1.9 | Yes | No | -| 2.0 | Yes | Yes | -| 2.1 | Yes | Yes | -| 2.2 | No | Yes | +## Batching -## Neo4j feature support +Although this need is rare, you can make multiple Cypher queries in a single network request, by passing a `queries` *array* rather than a single `query` string. + +Query `params` (and optionally `lean`) are then specified *per query*, so the elements in the array are `{query, params[, lean]}` objects. (Other options like `headers` remain "global" for the entire request.) + +```js +db.cypher({ + queries: [{ + query: 'MATCH (user:User {email: {email}}) RETURN user', + params: { + email: 'alice@example.com', + }, + }, { + query: 'MATCH (task:WorkerTask) RETURN task', + lean: true, + }, { + query: 'MATCH (task:WorkerTask) DELETE task', + }], + headers: { + 'X-Request-ID': '1234567890', + }, +}, callback); +``` + +The callback then receives an *array* of query results, one per query. + +```js +function callback(err, batchResults) { + if (err) throw err; + + var userResults = batchResults[0]; + var taskResults = batchResults[1]; + var deleteResults = batchResults[2]; + + // User results: + var userResult = userResults[0]; + if (!userResult) { + console.log('No user found.'); + } else { + var user = userResult['user']; + console.log('User %s (%s) found.', user._id, user.properties.name); + } + + // Worker task results: + if (!taskResults.length) { + console.log('No worker tasks to process.'); + } else { + taskResults.forEach(function (taskResult) { + var task = taskResult['task']; + console.log('Processing worker task %s...', task.operation); + }); + } + + // Delete results (shouldn’t have returned any): + assert.equal(deleteResults.length, 0); +}; +``` + +Importantly, batch queries execute (a) sequentially and (b) transactionally: either they all succeed, or they all fail. If you don't need them to be transactional, it can often be better to parallelize separate `db.cypher` calls instead. -| **Feature** | **Ver 1.x** | **Ver 2.x** | -|----------------------|-------------|-------------| -| Auth | No | Yes | -| Remote Cypher | Yes | Yes | -| Transactions | No | No | -| High Availability | No | No | -| Embedded JVM support | No | No | - + ## Changes @@ -151,8 +273,6 @@ This library is licensed under the [Apache License, Version 2.0][license]. -[neo4j]: http://neo4j.org/ -[what-is-a-graph-database]: http://neo4j.com/developer/graph-database/ [node.js]: http://nodejs.org/ [neo4j-rest-api]: http://docs.neo4j.org/chunked/stable/rest-api.html From 438d38382c68229abfa8af3a804fcbabcc1c3517 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Sun, 30 Aug 2015 11:46:05 -0700 Subject: [PATCH 080/121] v2 / Docs: document transactions! [skip ci] --- README.md | 104 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/README.md b/README.md index 091248c..01db8e9 100644 --- a/README.md +++ b/README.md @@ -244,6 +244,110 @@ function callback(err, batchResults) { Importantly, batch queries execute (a) sequentially and (b) transactionally: either they all succeed, or they all fail. If you don't need them to be transactional, it can often be better to parallelize separate `db.cypher` calls instead. +## Transactions + +You can also batch multiple Cypher queries into a single transaction across *multiple* network requests. This can be useful when application logic needs to run in between related queries. The queries will all succeed or fail together. + +To do this, begin a new transaction, make Cypher queries within that transaction, and then ultimately commit the transaction or roll it back. + +```js +var tx = db.beginTransaction(); + +function makeFirstQuery() { + tx.cypher({ + query: '...', + params {...}, + }, makeSecondQuery); +} + +function makeSecondQuery(err, results) { + if (err) throw err; + // ...some application logic... + tx.cypher({ + query: '...', + params: {...}, + }, finish); +} + +function finish(err, results) { + if (err) throw err; + // ...some application logic... + tx.commit(done); // or tx.rollback(done); +} + +function done(err, results) { + if (err) throw err; + // At this point, the transaction has been committed. +} + +makeFirstQuery(); +``` + +The transactional `cypher` method supports everything the normal `cypher` method does (e.g. `lean`, `headers`, and batch `queries`). In addition, you can pass `commit: true` to auto-commit the transaction (and save a network request) if the query succeeds. + +```js +function makeSecondQuery(err, results) { + if (err) throw err; + // ...some application logic... + tx.cypher({ + query: '...', + params: {...}, + commit: true, + }, done); +} + +function done(err, results) { + if (err) throw err; + // At this point, the transaction has been committed. +} +``` + +Importantly, transactions allow only one query at a time. To help preempt errors, you can inspect the state of the transaction, e.g. whether it's open for queries or not. + +```js +// Initially, transactions are open: +assert.equal(tx.state, tx.STATE_OPEN); + +// Making a query... +tx.cypher({ + query: '...', + params: {...}, +}, callback) + +// ...will result in the transaction being pending: +assert.equal(tx.state, tx.STATE_PENDING); + +// All other operations (making another query, committing, etc.) +// are rejected while the transaction is pending: +assert.throws(tx.renew.bind(tx)) + +function callback(err, results) { + // When the query returns, the transaction is likely open again, + // but it could be committed if `commit: true` was specified, + // or it could have been rolled back automatically (by Neo4j) + // if there was an error: + assert.notEqual([ + tx.STATE_OPEN, tx.STATE_COMMITTED, tx.STATE_ROLLED_BACK + ].indexOf(tx.state), -1); // i.e. tx.state is in this array +} +``` + +Finally, open transactions expire after some period of inactivity. (TODO: Link to manual.) This period is configurable in Neo4j, but it defaults to 60 seconds today. Transactions renew automatically on every query, but if you need to, you can inspect transactions' expiration times and renew them manually. + +```js +// Only open transactions (not already expired) can be renewed: +assert.equal(tx.state, tx.STATE_OPEN); +assert.notEqual(tx.state, tx.STATE_EXPIRED); + +console.log('Before:', tx.expiresAt, '(in', tx.expiresIn, 'ms)'); +tx.renew(function (err) { + if (err) throw err; + console.log('After:', tx.expiresAt, '(in', tx.expiresIn, 'ms)'); +}); +``` + +TODO: State diagram! + ## Basics @@ -190,7 +192,7 @@ Other options: ## Batching -Although this need is rare, you can make multiple Cypher queries in a single network request, by passing a `queries` *array* rather than a single `query` string. +You can also make multiple Cypher queries within a single network request, by passing a `queries` *array* rather than a single `query` string. Query `params` (and optionally `lean`) are then specified *per query*, so the elements in the array are `{query, params[, lean]}` objects. (Other options like `headers` remain "global" for the entire request.) @@ -252,7 +254,7 @@ Importantly, batch queries execute (a) **sequentially** and (b) **transactionall ## Transactions -You can also batch multiple Cypher queries into a single transaction across *multiple* network requests. This can be useful when application logic needs to run in between related queries. The queries will all succeed or fail together. +You can also batch multiple Cypher queries into a single transaction across *multiple* network requests. This can be useful when application logic needs to run in between related queries (e.g. for domain-aware cascading deletes), or Neo4j state needs to be coordinated with side effects (e.g. writes to another data store). The queries will all succeed or fail together. To do this, begin a new transaction, make Cypher queries within that transaction, and then ultimately commit the transaction or roll it back. @@ -553,7 +555,7 @@ req.on('end', function () { ## Errors -(TODO) +To achieve robustness in your app, it's vitally important to handle errors precisely. ## Tuning From f68eafb9720ac934c0346a10820cb9bd4d00731e Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 09:33:29 -0400 Subject: [PATCH 105/121] v2 / Docs: write v1 migration guide! [skip ci] --- CHANGELOG.md | 74 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 71 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a08212..7b30997 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,11 +4,11 @@ node-neo4j v2 is a ground-up rewrite of node-neo4j, partly to take advantage of node-neo4j v2's big new features and changes are: -- An emphasis on **Cypher** now (renamed from ~~`db.query`~~ to `db.cypher`); no more ~~individual CRUD methods~~ (like v1's ~~`db.createNode`~~, etc.). For anything non-trivial, Cypher is much more expressive and robust, so writing it directly is highly encouraged. +- An emphasis on **Cypher** now (no more individual CRUD methods like v1's `db.createNode`). For anything non-trivial, Cypher is much more robust and expressive, so writing it directly is highly encouraged. - Support for Cypher **transactions** and **batching**, as well as arbitrary **HTTP requests** (including streaming support) and **custom headers**. -- First-class support for Neo4j 2.0 **labels**, **schema indexes**, and **constraints**, as well as Neo4j 2.2's **password auth**. +- First-class support for Neo4j 2.0 **labels**, **schema indexes**, and **constraints**, as well as Neo4j 2.2's **password-based auth**. - Much better **error handling**, differentiating between **client**, **server**, and **transient** errors. @@ -20,7 +20,75 @@ For details on all these features and more, be sure to read through the **[readm ### Migrating from v1 -TODO +If you're currently running node-neo4j v1, you may have to make some significant changes to your code, but hopefully the above features make it worth it. =) + +Simple changes: + +- `db.query(query, params, callback)` is now `db.cypher({query, params}, callback)` + +- `node.id` and `rel.id` are now `node._id` and `rel._id` + + - This is because using these Neo4j internal IDs is officially discouraged now. Better to use a unique property instead (e.g. `username` or `uuid`). + +- `node.data` and `rel.data` are now `node.properties` and `rel.properties` + +Removed CRUD methods (use Cypher instead): + +- `db.createNode` and `node.createRelationshipTo`/`From` + +- `db.getNodeById` and `db.getRelationshipById` + +- `node.getRelationships` and `node.getRelationshipNodes` + +- `node.incoming`, `node.outgoing`, `node.all`, and `node.path` + +- `node.save` and `rel.save` + + - In general, `Node` and `Relationship` instances are no longer "stateful". Instead of making changes to the database by modifying properties on these instances and calling `save`, just make changes via Cypher directly. Much more robust, precise, and expressive. Note in particular that [`SET node += {props}`](http://neo4j.com/docs/stable/query-set.html#set-adding-properties-from-maps) lets you update some properties without overwriting others. + +- `node.exists` and `rel.exists` + + - Since `Node` and `Relationship` instances are no longer stateful, node-neo4j v2 only ever returns instances for data returned from Neo4j. So these nodes and relationships _always_ "exist" (at least, at the time they're returned). + +- `node.del(ete)` and `rel.del(ete)` + +Removed legacy index management (use schema indexes instead): + +- `db.createNodeIndex` and `db.createRelationshipIndex` + +- `db.getNodeIndexes` and `db.getRelationshipIndexes` + +- `db.getIndexedNode(s)` and `db.getIndexedRelationship(s)` + +- `db.queryNodeIndex` and `db.queryRelationshipIndex` + +- `db.deleteNodeIndex` and `db.deleteRelationshipIndex` + +- `node.(un)index` and `rel.(un)index` + +Removed miscellaneous: + +- `rel.start` and `rel.end` + + - These used to be full Node instances, but it's no longer efficient to load full node data for all relationships by default, so only `rel._fromId` and `rel._toId` exist now. Expand your Cypher query to return full nodes if you need them. + +- `Path` class + + - For similar reasons as `rel.start` and `rel.end`. Expand your Cypher query to return [`NODES(path)`](http://neo4j.com/docs/stable/query-functions-collection.html#functions-nodes) or [`RELATIONSHIPS(path)`](http://neo4j.com/docs/stable/query-functions-collection.html#functions-relationships) if you need them. + +- `db.execute` (for [Gremlin](http://gremlin.tinkerpop.com/) scripts) + + - Gremlin has been dropped as a default plugin in Neo4j 2.x, and Cypher is clearly the recommended query language going forward. If you do need Gremlin support, you can always make [HTTP requests](./README.md#http--plugins) to the [Gremlin endpoint](https://github.com/thinkaurelius/neo4j-gremlin-plugin) directly. + +- `db.reviveJSON` and `db.fromJSON` + + - We may add these back if needed, but for now, since `Node` and `Relationship` instances are no longer stateful (they don't even have any public methods), reviving JSON isn't really needed. You can still `JSON.stringify` nodes and relationships, and you should be able to use parsed JSON objects directly. + +- [Streamline futures](https://github.com/Sage/streamlinejs#futures) (TODO: Promises instead?) + + - This library is no longer written in Streamline, so methods no longer return Streamline futures. If your app isn't written in Streamline, this likely never mattered to you. But even if your app _is_ written in Streamline, this change may not matter to you, as Streamline 1.0 lets you call [_any_ async function](https://github.com/Sage/streamlinejs/issues/181) with `!_`. + +**[node-neo4j-template PR #18](https://github.com/aseemk/node-neo4j-template/pull/18)** is a good model for the changes needed (most notably commit [`bbf8e86`](https://github.com/aseemk/node-neo4j-template/pull/18/commits/bbf8e865d99888bdfeed86c61ea5f5f6ad611981)). Take a look through that, and if you run into any issues migrating your own code, feel free to [reach out for help](./README.md#help). Good luck! ## Version 1.x.x From 6b740c6a9e419a898990901868303cd1ead672ce Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:20:56 -0400 Subject: [PATCH 106/121] v2 / Docs: WIP [skip ci] --- README.md | 56 ++++++++++++------------------------------------------- 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/README.md b/README.md index bb360d5..a2d3ffd 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ Yields e.g.: } ``` -See [node-neo4j-template](https://github.com/aseemk/node-neo4j-template) for a more thorough example. +See **[node-neo4j-template](https://github.com/aseemk/node-neo4j-template)** for a more thorough example. @@ -354,9 +354,9 @@ tx.renew(function (err) { }); ``` -The full [state diagram](https://mix.fiftythree.com/aseemk/2462211) putting this all together: +The full [state diagram](https://mix.fiftythree.com/aseemk/10779878) putting this all together: -[![Neo4j transaction state diagram](https://d3ayzibdlq49a1.cloudfront.net/f2a67a92-8b73-44ba-bb29-0ce12069c57e/image/f2a67a92-8b73-44ba-bb29-0ce12069c57e_image_2048x1536.png)](https://mix.fiftythree.com/aseemk/2462211) +[![Neo4j transaction state diagram](https://blobs-public.fiftythree.com/9LdWt0fwPjeT_o0nZ6b3o1w2qCwKs6NuNGZ4d3db86UKp2r7)](https://mix.fiftythree.com/aseemk/10779878) ## Headers @@ -573,57 +573,25 @@ To achieve robustness in your app, it's vitally important to handle errors preci ## Help -(TODO) - - -## Contributing - -[See CONTRIBUTING.md »](./CONTRIBUTING.md) - - - - +[See CONTRIBUTING.md »](./CONTRIBUTING.md) -## Changes +## History -See the [Changelog][changelog] for the full history of changes and releases. +[See CHANGELOG.md »](./CHANGELOG.md) ## License -This library is licensed under the [Apache License, Version 2.0][license]. - - - -[node.js]: http://nodejs.org/ -[neo4j-rest-api]: http://docs.neo4j.org/chunked/stable/rest-api.html - -[api-docs]: http://coffeedoc.info/github/thingdom/node-neo4j/master/ -[aseemk]: https://github.com/aseemk -[node-neo4j-template]: https://github.com/aseemk/node-neo4j-template -[semver]: http://semver.org/ - -[coffeescript]: http://coffeescript.org/ -[streamline.js]: https://github.com/Sage/streamlinejs - -[changelog]: CHANGELOG.md -[issue-tracker]: https://github.com/thingdom/node-neo4j/issues -[license]: http://www.apache.org/licenses/LICENSE-2.0.html -[google-group]: https://groups.google.com/group/node-neo4j +Copyright © 2016 Aseem Kishore and [contributors](https://github.com/thingdom/node-neo4j/graphs/contributors). -[stackoverflow-ask]: http://stackoverflow.com/questions/ask?tags=node.js,neo4j,thingdom +This library is licensed under the [Apache License, Version 2.0](./LICENSE). From af8da3c7c5e81023684717037ff76003562ce7ee Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:21:00 -0400 Subject: [PATCH 107/121] v2 / Docs: fill in license copyright. [skip ci] --- LICENSE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LICENSE b/LICENSE index d645695..3a3f200 100644 --- a/LICENSE +++ b/LICENSE @@ -187,7 +187,7 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright [yyyy] [name of copyright owner] + Copyright 2016 Aseem Kishore and other contributors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. From bab340aa7f683966e6ad4469ed5872a9639908dd Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:24:17 -0400 Subject: [PATCH 108/121] fix docs [skip ci] --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a2d3ffd..b5e9ac7 100644 --- a/README.md +++ b/README.md @@ -582,16 +582,16 @@ You can also try **[Gitter](https://gitter.im/thingdom/node-neo4j)**, **[Stack O ## Contributing -[See CONTRIBUTING.md »](./CONTRIBUTING.md) +[See **CONTRIBUTING.md** »](./CONTRIBUTING.md) ## History -[See CHANGELOG.md »](./CHANGELOG.md) +[See **CHANGELOG.md** »](./CHANGELOG.md) ## License -Copyright © 2016 Aseem Kishore and [contributors](https://github.com/thingdom/node-neo4j/graphs/contributors). +Copyright © 2016 **[Aseem Kishore](https://github.com/aseemk)** and [contributors](https://github.com/thingdom/node-neo4j/graphs/contributors). -This library is licensed under the [Apache License, Version 2.0](./LICENSE). +This library is licensed under the **[Apache License, Version 2.0](./LICENSE)**. From 70ad8fe47c5956dd59c9c5b31a411ab756ed234a Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:32:19 -0400 Subject: [PATCH 109/121] fix docs [skip ci] --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b5e9ac7..128d555 100644 --- a/README.md +++ b/README.md @@ -354,9 +354,9 @@ tx.renew(function (err) { }); ``` -The full [state diagram](https://mix.fiftythree.com/aseemk/10779878) putting this all together: +The full [state diagram](https://paper.fiftythree.com/aseemk/10779878) putting this all together: -[![Neo4j transaction state diagram](https://blobs-public.fiftythree.com/9LdWt0fwPjeT_o0nZ6b3o1w2qCwKs6NuNGZ4d3db86UKp2r7)](https://mix.fiftythree.com/aseemk/10779878) +[![Neo4j transaction state diagram](https://blobs-public.fiftythree.com/lLjEolaK8srcsvhkvy7n5DwMcJJTFiFe5e2sBrxOTzE1TZdj)](https://paper.fiftythree.com/aseemk/10779878) ## Headers From 01ead440ee41be333aeb196565018cb4e2ca2572 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:34:12 -0400 Subject: [PATCH 110/121] fix docs [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 128d555..1378ec5 100644 --- a/README.md +++ b/README.md @@ -356,7 +356,7 @@ tx.renew(function (err) { The full [state diagram](https://paper.fiftythree.com/aseemk/10779878) putting this all together: -[![Neo4j transaction state diagram](https://blobs-public.fiftythree.com/lLjEolaK8srcsvhkvy7n5DwMcJJTFiFe5e2sBrxOTzE1TZdj)](https://paper.fiftythree.com/aseemk/10779878) +Neo4j transaction state diagram ## Headers From 11b86182f0226eff70d0d6ea48b585609a761865 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Tue, 5 Apr 2016 15:35:51 -0400 Subject: [PATCH 111/121] fix docs [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1378ec5..cb5eb2f 100644 --- a/README.md +++ b/README.md @@ -356,7 +356,7 @@ tx.renew(function (err) { The full [state diagram](https://paper.fiftythree.com/aseemk/10779878) putting this all together: -Neo4j transaction state diagram +Neo4j transaction state diagram ## Headers From 193311e547c0d5b0c77eadd7db1e357500133ea0 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:15:51 -0400 Subject: [PATCH 112/121] fix docs [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index cb5eb2f..3985024 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ db.cypher({ }, callback); ``` -It's extremely important to **pass `params` separately**. If you concatenate them into the `query`, you'll be vulnerable to injection attacks, and Neo4j performance will suffer as well. +It's extremely important to **pass `params` separately**. If you concatenate them into the `query`, you'll be vulnerable to injection attacks, and Neo4j performance will suffer as well. _(Note that parameters can't be used for labels, property names, and relationship types, as those things determine the query plan. [Docs »](http://neo4j.com/docs/stable/cypher-parameters.html))_ Cypher queries *always* return a list of results (like SQL rows), with each result having common properties (like SQL columns). Thus, query **results** passed to the callback are *always* an **array** (even if it's empty), and each **result** in the array is *always* an **object** (even if it's empty). From 0a14a874006eced5c697623d152c804b0d0edba7 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:16:09 -0400 Subject: [PATCH 113/121] v2 / Docs: fill out errors section! [skip ci] --- README.md | 121 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3985024..337f3de 100644 --- a/README.md +++ b/README.md @@ -555,7 +555,126 @@ req.on('end', function () { ## Errors -To achieve robustness in your app, it's vitally important to handle errors precisely. +To achieve robustness in your app, it's vitally important to [handle errors precisely](http://www.joyent.com/developers/node/design/errors). Neo4j supports this nicely by returning semantic and precise [error codes](http://neo4j.com/docs/stable/status-codes.html). + +There are multiple levels of detail, but the high-level classifications are a good granularity for decision-making: + +- `ClientError` (likely a bug in your code, but possibly invalid user input) +- `DatabaseError` (a bug in Neo4j) +- `TransientError` (occasionally expected; can/should retry) + +node-neo4j translates these classifications to named `Error` subclasses. That means there are two ways to detect Neo4j errors: + +```js +// `instanceof` is recommended: +err instanceof neo4j.TransientError + +// `name` works too, though: +err.name === 'neo4j.TransientError' +``` + +These error instances also have a `neo4j` property with semantic data inside. E.g. Cypher errors have data of the form `{code, message}`: + +```json +{ + "code": "Neo.TransientError.Transaction.DeadlockDetected", + "message": "LockClient[83] can't wait on resource ...", + "stackTrace": "..." +} +``` + +Other types of errors (e.g. [managing schema](#management)) may have different forms of `neo4j` data: + +```json +{ + "exception": "BadInputException", + "fullname": "org.neo4j.server.rest.repr.BadInputException", + "message": "Unable to add label, see nested exception.", + "stackTrace": [...], + "cause": {...} +} +``` + +Finally, malformed (non-JSON) responses from Neo4j (rare) will have `neo4j` set to the raw response string, while native Node.js errors (e.g. DNS errors) will be propagated in their original form, to avoid masking unexpected issues. + +Putting all this together, you now have the tools to handle node-neo4j errors precisely! For example, we have helpers similar to these at FiftyThree: + +```js +// A query or transaction failed. Should we retry it? +// We check this in a retry loop, with proper backoff, etc. +// http://aseemk.com/talks/advanced-neo4j#/50 +function shouldRetry(err) { + // All transient errors are worth retrying, of course. + if (err instanceof neo4j.TransientError) { + return true; + } + + // If the database is unavailable, it's probably failing over. + // We expect it to come back up quickly, so worth retrying also. + if (isDbUnavailable(err)) { + return true; + } + + // There are a few other non-transient Neo4j errors we special-case. + // Important: this assumes we don't have bugs in our code that would trigger + // these errors legitimately. + if (typeof err.neo4j === 'object' && ( + // If a failover happened when we were in the middle of a transaction, + // the new instance won't know about that transaction, so we re-do it. + err.neo4j.code === 'Neo.ClientError.Transaction.UnknownId' || + // These are current Neo4j bugs we occasionally hit with our queries. + err.neo4j.code === 'Neo.ClientError.Statement.EntityNotFound' || + err.neo4j.code === 'Neo.DatabaseError.Statement.ExecutionFailure')) { + return true; + } + + return false; +} + +// Is this error due to Neo4j being down, failing over, etc.? +// This is a separate helper because we also retry less aggressively in this case. +function isDbUnavailable(err) { + // If we're unable to connect, we see these particular Node.js errors. + // https://nodejs.org/api/errors.html#errors_common_system_errors + // E.g. http://stackoverflow.com/questions/17245881/node-js-econnreset + if ((err.syscall === 'getaddrinfo' && err.code === 'ENOTFOUND') || + (err.syscall === 'connect' && err.code === 'EHOSTUNREACH') || + (err.syscall === 'read' && err.code === 'ECONNRESET')) { + return true; + } + + // We load balance via HAProxy, so if Neo4j is unavailable, HAProxy responds + // with 5xx status codes. + // node-neo4j sees this and translates to a DatabaseError, but the body is + // HTML, not JSON, so the `neo4j` property is simply the HTML string. + if (err instanceof neo4j.DatabaseError && typeof err.neo4j === 'string' && + err.neo4j.match(/(502 Bad Gateway|503 Service Unavailable)/)) { + return true; + } + + return false; +} +``` + +In addition to all of the above, node-neo4j also embeds the most useful information from `neo4j` into the `message` and `stack` properties, so you don't need to do anything special to log meaningful, actionable, and debuggable errors. (E.g. Node's native logging of errors, both via `console.log` and on uncaught exceptions, includes this info.) + +Here are some example snippets from real-world `stack` traces: + +``` +neo4j.ClientError: [Neo.ClientError.Statement.ParameterMissing] Expected a parameter named email + +neo4j.ClientError: [Neo.ClientError.Schema.ConstraintViolation] Node 15 already exists with label User and property "email"=[15] + +neo4j.DatabaseError: [Neo.DatabaseError.Statement.ExecutionFailure] scala.MatchError: (email,null) (of class scala.Tuple2) + at + at + +neo4j.DatabaseError: 502 Bad Gateway response for POST /db/data/transaction/commit: \"

502 Bad Gateway

\\nThe server returned an invalid or incomplete response.\\n\\n\" + +neo4j.TransientError: [Neo.TransientError.Transaction.DeadlockDetected] LockClient[1150] can't wait on resource RWLock[NODE(196), hash=2005718009] since => LockClient[1150] <-[:HELD_BY]- RWLock[NODE(197), hash=1180589294] <-[:WAITING_FOR]- LockClient[1149] <-[:HELD_BY]- RWLock[NODE(196), hash=2005718009] +``` + +Precise and helpful error reporting is one of node-neo4j's best strengths. We hope it helps your app run smoothly! ## Tuning From 1e647bbc0500c20e7bc85c7e8c44e2b801457ea1 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:16:48 -0400 Subject: [PATCH 114/121] v2 / Tests: improve transaction cleanup some more. --- test-new/transactions._coffee | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index ed9b2e5..70d9bef 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -32,20 +32,22 @@ beginTx = -> # The default transaction expiry time of 60 seconds is also far too long. # So we track all transactions we create (see above), and this method can be # called to clean those transactions up whenever needed. -# It *should* be called at the end at least (see test near end of suite below). cleanupTxs = (_) -> - while tx = OPEN_TXS.pop() - switch tx.state - when tx.STATE_COMMITTED, tx.STATE_ROLLED_BACK, tx.STATE_EXPIRED - continue - when tx.STATE_PENDING - throw new Error 'Unexpected: transaction state is pending! - Maybe a test timed out mid-request?' - when tx.STATE_OPEN - tx.rollback _ - expect(tx.state).to.equal tx.STATE_ROLLED_BACK - else - throw new Error "Unrecognized tx state! #{tx.state}" + # We parallelize these rollbacks to keep test performance fast: + flows.collect _, + while tx = OPEN_TXS.pop() then do (tx, _=!_) -> + switch tx.state + when tx.STATE_COMMITTED, tx.STATE_ROLLED_BACK, tx.STATE_EXPIRED + return + when tx.STATE_PENDING + throw new Error 'Unexpected: transaction state is pending! + Maybe a test timed out mid-request? + Can’t rollback since concurrent reqs not allowed...' + when tx.STATE_OPEN + tx.rollback _ + expect(tx.state).to.equal tx.STATE_ROLLED_BACK + else + throw new Error "Unrecognized tx state! #{tx.state}" # Calls the given asynchronous function with a placeholder callback, and # immediately returns a "future" that can be called with a real callback. @@ -89,6 +91,9 @@ expectTxErrorRolledBack = (tx, _) -> describe 'Transactions', -> + afterEach (_) -> + cleanupTxs _ + it 'should support simple queries', (_) -> tx = beginTx() @@ -717,8 +722,5 @@ describe 'Transactions', -> it 'should support streaming (TODO)' - it '(clean up open txs)', (_) -> - cleanupTxs _ - it '(delete test graph)', (_) -> fixtures.deleteTestGraph module, _ From 5e46e5c0449006c87c395899965ac8cb168651c9 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:28:12 -0400 Subject: [PATCH 115/121] v2 / Errors: improve Neo4j Java stack traces. --- lib-new/errors.coffee | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 516d2dd..681c78b 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -80,11 +80,12 @@ class @Error extends Error stackTrace = "#{message}: #{stackTrace}" # Stack traces can include "Caused by" lines which aren't indented, - # and indented lines use tabs. Normalize to 4 spaces, and indent - # everything one extra level, to differentiate from Node.js lines. - stackTrace = stackTrace - .replace /\t/g, ' ' - .replace /\n/g, '\n ' + # and indented lines use tabs. Normalize to 4 spaces (like Node), + # and add a divider to differentiate Neo4j lines from Node.js ones. + # Note that the Neo4j stack often ends in a newline already. + stackTrace = stackTrace.replace /\t/g, ' ' + stackTrace += '\n' if stackTrace[-1..] isnt '\n' + stackTrace += ' <<< Neo4j stack above; Node.js stack below >>>' fullMessage += stackTrace From ed15bf4e555fb5c9f93425cb8a15a97dc9240726 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:29:03 -0400 Subject: [PATCH 116/121] v2 / Errors: mask low-level stack a bit more. --- lib-new/errors.coffee | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 681c78b..14a6dbb 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -4,16 +4,16 @@ http = require 'http' class @Error extends Error - constructor: (@message='Unknown error', @neo4j={}) -> + constructor: (@message='Unknown error', @neo4j={}, _stackStart) -> @name = 'neo4j.' + @constructor.name - Error.captureStackTrace @, @constructor + Error.captureStackTrace @, _stackStart # # Accepts the given HTTP client response, and if it represents an error, # creates and returns the appropriate Error instance from it. # If the response doesn't represent an error, returns null. # - @_fromResponse: (resp) -> + @_fromResponse: (resp, _stackStart=arguments.callee) -> {body, headers, statusCode} = resp return null if statusCode < 400 @@ -23,7 +23,7 @@ class @Error extends Error # TODO: Is it possible to get back more than one error? # If so, is it fine for us to just use the first one? [error] = body.errors - return @_fromObject error + return @_fromObject error, _stackStart # TODO: Do some status codes (or perhaps inner `exception` names) # signify Transient errors rather than Database ones? @@ -44,13 +44,13 @@ class @Error extends Error if logBody and body? message += ": #{JSON.stringify body, null, 4}" - new ErrorClass message, body + new ErrorClass message, body, _stackStart # # Accepts the given (Neo4j v2) error object, and creates and returns the # appropriate Error instance for it. # - @_fromObject: (obj) -> + @_fromObject: (obj, _stackStart=arguments.callee) -> # NOTE: Neo4j 2.2 seems to return both `stackTrace` and `stacktrace`. # https://github.com/neo4j/neo4j/issues/4145#issuecomment-78203290 # Normalizing to consistent `stackTrace` before we parse below! @@ -93,7 +93,7 @@ class @Error extends Error else fullMessage += message - new ErrorClass fullMessage, obj + new ErrorClass fullMessage, obj, _stackStart # TODO: Helper to rethrow native/inner errors? Not sure if we need one. From eecc64d62af0d3a01a739f99ee321a866a511613 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:29:38 -0400 Subject: [PATCH 117/121] Revert "v2 / Errors: mask low-level stack a bit more." MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This reverts commit ed15bf4e555fb5c9f93425cb8a15a97dc9240726. It’s probably better to be honest/transparent. --- lib-new/errors.coffee | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/lib-new/errors.coffee b/lib-new/errors.coffee index 14a6dbb..681c78b 100644 --- a/lib-new/errors.coffee +++ b/lib-new/errors.coffee @@ -4,16 +4,16 @@ http = require 'http' class @Error extends Error - constructor: (@message='Unknown error', @neo4j={}, _stackStart) -> + constructor: (@message='Unknown error', @neo4j={}) -> @name = 'neo4j.' + @constructor.name - Error.captureStackTrace @, _stackStart + Error.captureStackTrace @, @constructor # # Accepts the given HTTP client response, and if it represents an error, # creates and returns the appropriate Error instance from it. # If the response doesn't represent an error, returns null. # - @_fromResponse: (resp, _stackStart=arguments.callee) -> + @_fromResponse: (resp) -> {body, headers, statusCode} = resp return null if statusCode < 400 @@ -23,7 +23,7 @@ class @Error extends Error # TODO: Is it possible to get back more than one error? # If so, is it fine for us to just use the first one? [error] = body.errors - return @_fromObject error, _stackStart + return @_fromObject error # TODO: Do some status codes (or perhaps inner `exception` names) # signify Transient errors rather than Database ones? @@ -44,13 +44,13 @@ class @Error extends Error if logBody and body? message += ": #{JSON.stringify body, null, 4}" - new ErrorClass message, body, _stackStart + new ErrorClass message, body # # Accepts the given (Neo4j v2) error object, and creates and returns the # appropriate Error instance for it. # - @_fromObject: (obj, _stackStart=arguments.callee) -> + @_fromObject: (obj) -> # NOTE: Neo4j 2.2 seems to return both `stackTrace` and `stacktrace`. # https://github.com/neo4j/neo4j/issues/4145#issuecomment-78203290 # Normalizing to consistent `stackTrace` before we parse below! @@ -93,7 +93,7 @@ class @Error extends Error else fullMessage += message - new ErrorClass fullMessage, obj, _stackStart + new ErrorClass fullMessage, obj # TODO: Helper to rethrow native/inner errors? Not sure if we need one. From 384f069832d9eb1416f99c7b07c0856af1144aa3 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:37:31 -0400 Subject: [PATCH 118/121] fix error docs [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 337f3de..fdb4abb 100644 --- a/README.md +++ b/README.md @@ -597,7 +597,7 @@ Other types of errors (e.g. [managing schema](#management)) may have different f Finally, malformed (non-JSON) responses from Neo4j (rare) will have `neo4j` set to the raw response string, while native Node.js errors (e.g. DNS errors) will be propagated in their original form, to avoid masking unexpected issues. -Putting all this together, you now have the tools to handle node-neo4j errors precisely! For example, we have helpers similar to these at FiftyThree: +Putting all this together, you now have the tools to handle Neo4j errors precisely! For example, we have helpers similar to these at FiftyThree: ```js // A query or transaction failed. Should we retry it? From d74ae498316184792442f9d9cedf3434a049d5b8 Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:42:11 -0400 Subject: [PATCH 119/121] fix improve transaction cleanup --- test-new/transactions._coffee | 3 +++ 1 file changed, 3 insertions(+) diff --git a/test-new/transactions._coffee b/test-new/transactions._coffee index 70d9bef..759a434 100644 --- a/test-new/transactions._coffee +++ b/test-new/transactions._coffee @@ -35,7 +35,10 @@ beginTx = -> cleanupTxs = (_) -> # We parallelize these rollbacks to keep test performance fast: flows.collect _, + # Streamline futures use `!` rather than `not` by convention. + # coffeelint: disable=prefer_english_operator while tx = OPEN_TXS.pop() then do (tx, _=!_) -> + # coffeelint: enable=prefer_english_operator switch tx.state when tx.STATE_COMMITTED, tx.STATE_ROLLED_BACK, tx.STATE_EXPIRED return From 5169575f1e2a1c0730992a09ed025d50ac790a9d Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 00:56:18 -0400 Subject: [PATCH 120/121] v2 / Tests: no longer allow Neo4j 2.0 failures; remove io.js. --- .travis.yml | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/.travis.yml b/.travis.yml index c20e92e..8f0e5ab 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,9 +1,9 @@ language: node_js node_js: + # test across multiple versions of Node: - "5" - "4" - - "io.js" - "0.12" - "0.10" @@ -17,12 +17,6 @@ env: - NEO4J_VERSION="2.1.8" - NEO4J_VERSION="2.0.4" -matrix: - # but we may want to allow our tests to fail against *some* Neo4j versions, - # e.g. due to unstability, bugs, or breaking changes for our test code. - allow_failures: - - env: NEO4J_VERSION="2.0.4" # seems to have transaction bugs - before_install: # install Neo4j locally: - wget dist.neo4j.org/neo4j-community-$NEO4J_VERSION-unix.tar.gz From b730d8367a1363fc11eb5c21af1466956a28b31c Mon Sep 17 00:00:00 2001 From: Aseem Kishore Date: Wed, 13 Apr 2016 08:32:37 -0400 Subject: [PATCH 121/121] fix error docs [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fdb4abb..bc5885a 100644 --- a/README.md +++ b/README.md @@ -669,7 +669,7 @@ neo4j.DatabaseError: [Neo.DatabaseError.Statement.ExecutionFailure] scala.MatchE at at -neo4j.DatabaseError: 502 Bad Gateway response for POST /db/data/transaction/commit: \"

502 Bad Gateway

\\nThe server returned an invalid or incomplete response.\\n\\n\" +neo4j.DatabaseError: 502 Bad Gateway response for POST /db/data/transaction/commit: "

502 Bad Gateway

\nThe server returned an invalid or incomplete response.\n\n" neo4j.TransientError: [Neo.TransientError.Transaction.DeadlockDetected] LockClient[1150] can't wait on resource RWLock[NODE(196), hash=2005718009] since => LockClient[1150] <-[:HELD_BY]- RWLock[NODE(197), hash=1180589294] <-[:WAITING_FOR]- LockClient[1149] <-[:HELD_BY]- RWLock[NODE(196), hash=2005718009] ```