Hi Everyone,
Here is an update on the position of the OpenReferral UK Validator for the profiles. While my primary focus has been the UK requirements, I have tried to keep in mind, and satisfy where possible, international requirements too.
Consolidated UK Profiles
One step that we have taken, is to consolidate our schema files into a single location. Previously there were a “few” copies in various repositories, floating around Github. If you have a thumb drive with a UK profile on it, it’s out of date. 
Requirements Status Report
The requirements here, were supplied to me in October, by Matt Marshall.
Source: Requirement Spreadsheet
| # |
Requirement |
Status |
Implementation Details |
| 1 |
JSON Schema validation via HSDS |
 |
IJsonValidatorService uses Newtonsoft.Json.Schema. |
| 2 |
Endpoint discovery from root URL |
|
IOpenApiDiscoveryService parses the OpenAPI spec to extract paths. |
| 3 |
Remote data retrieval via HTTP |
|
ExecuteHttpRequestAsync() uses injected HttpClient. |
| 4 |
Determine appropriate schema |
|
Extracts version from feed; defaults to HSDS-UK-1 if missing. |
| 5 |
Accept optional Auth tokens |
|
The validator accepts in the body of the request an datasource auth object with several options. |
| 6 |
Use bearer tokens for requests |
|
As above. |
| 7 |
Supply parameters to API (Testing) |
 |
Currently limited to pagination parameters. |
| 8 |
Validate results against parameters |
|
Currently limited to pagination validation. |
| 9 |
Report HTTP status of endpoints |
|
Returns detailed HttpTestResult (StatusCode, errors, etc.). |
| 10 |
Manual Profile override |
|
OpenApiSchemaUrl option allows users to skip auto-discovery. Also supports validation if the schema requires it. |
| 11 |
JSON report of pass/fail fields |
|
Reports failures and full JSON, but does not list “passed” fields. |
| 12 |
Detail reason for field failure |
|
ValidationError includes Path, Message, Code, and LineNumber. |
| 13 |
Highlight undefined fields |
 |
Not yet flagging fields present in data but missing from schema. |
| 14 |
Local/Community installation |
|
Containerized .NET application with Docker/Docker-Compose. |
Open Questions
Requirement 4: Schema Resolution (The “UK Problem”)
There remains an issue (within ORUK at least), that data feeds don’t always provide the correct URL to the deployed version of the schema. Version 1 feeds (UK), don’t necessarily provide any metadata identifying the version. With that in mind, I have (for now) had to assume HSDS-UK-1 if there is no direct specification. Most V3 feeds list the profile as “HSDS-UK-3.0”, making my job easier…except 1! I would like to resolve this, before I leave the project.
- Do we expect the data feeds to identify the schema uri to validate against?
- Would we trust it?
- Proposed Resolution: Maintain an Enum or DB of known profiles while keeping the manual override option.
- Questions: Do international feeds suffer from this?
- Should we implement a “whitelist” of acceptable metadata URLs?
Requirement 5 & 6: Authentication
I have now satisfied this requirement, for both requests to the data source, and the schema. I added the schema authentication because our staging environment is not publicly accessible.
Requirement 11: Reporting “Passed” Fields
Currently, we only return failures.
- Question: Would a list of “passed” fields add value, or would it just create unnecessary bulk in the report?
Requirement 13: Undefined Fields
- Status: This is the only “Not Satisfied” item. I do not presently flag fields that exist in the payload but are absent from the schema definitions.
Next Steps
The application contains a docker image with instructions in the wiki, which could be used for easy local deployment.
Post Request Body Structure
Bellow is the “current” body format of the post request to the validator.
{
"openApiSchema": {
"url": "string",
"authentication": {
"apiKey": "",
"apiKeyHeader": "X-API-Key",
"bearerToken": "",
"basicAuth": {
"username": "",
"password": ""
},
"customHeaders": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
}
}
},
"baseUrl": "string",
"dataSourceAuth": {
"apiKey": "",
"apiKeyHeader": "X-API-Key",
"bearerToken": "",
"basicAuth": {
"username": "",
"password": ""
},
"customHeaders": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
}
},
"options": {
"testEndpoints": true,
"validateSpecification": true,
"timeoutSeconds": 30,
"maxConcurrentRequests": 5,
"skipAuthentication": true,
"testOptionalEndpoints": true,
"treatOptionalEndpointsAsWarnings": true,
"includeResponseBody": false,
"includeTestResults": true,
"returnRawResult": true
}
}
Notes:
- Leave returnRawResult: true. (I might rename it) False, with discard a lot of the analytics data and return a format expected by the ORUK Website.
- If you pass multiple validation options in the request, they will all be set on the http request to either the data source or the schema.
- I am going to remove skipAuthentication. It isn’t needed. If there are no values to set it doesn’t send them,.
- IncludeResponseBody: If true this will return the full response received for every endpoint test request to the data source.
I hope this is heading in the right direction.
Regards
Jeff
