I think we need this as we migrate UK publishers to the new structure and API which are not radically different from what we have in the UK now, but bring more international consistency. (Consultation on the UK profile is still needed before it is finalised.)
The validator and dashboard ultimately need to work for all future versions of the Standard and for different profiles. The UK has the most immediate need.
We’d welcome comments here and at the new working group meeting.
I wonder if the ORUK API Connector concept can fit into this. If I recall the idea with that was that a feed could be registered in the connector, then used by other tools such as the validator or a directory viewer app or in the future a merge/dedupe tool etc.
Thanks @devin This is the ORUK Connector. We removed it from our menus because we don’t (yet) have sufficient feeds and consuming tools that meet enough API rules for it to work in many circumstances.
The idea of the connector is to connect any API feed, such as Placecube’s Open Place Directory or TPX Impact’s Outpost to any consuming application such as the Local Government Association’s Service Finder or TPX Impact’s Scout. However, most front-end tools rely on the search filters for the /services web method being implemented so no local data storage is needed by the consuming application. That is quite rare.
Nevertheless, I think it’s work us exploring bringing back the Connector. The code is very easy to implement and configure.
So we’ve heard a lot of interest in an updated validator, and I’m going to try to accelerate the process from here.
I’m glad to hear @devin likes @MikeThacker’s proposal, and also I want to get more input from a broader swath of the community as to desired features and related design objectives.
One thing that might help me is updating this proposal to include a set of user stories that are specifically formatted in the standard way – so that we can clearly state our assumptions about who wants to do what. i.e. “As a [database administrator] I want to [take this action] in order to [receive this benefit].” Perhaps mike can help us articulate the user stories that are implicitly assumed by this proposal – and then we can do some polling to get input on which of them are the highest priority, and whether any important stories are missing.
We have some other big questions – such as whether this validator tool should be deployable code or a hosted service, and what language it should be developed in – but perhaps we’ll be better able to have those conversations by first more clearly articulating our assumptions about what the users need.
Hi all! Did my best to distill points in the proposal and this Validation Tools spreadsheet into a starting point for potential user stories. I still have a pretty loose grasp on things- especially the technical- so please feel free to edit as needed!
@bloom, @MikeThacker, and I recently had another go at the user stories-- similar rows were merged, estimated value + difficulty were added, as were indicators and notes commenting on which features are to be included in the UK tool.
Looping back here, we’re still iterating on the validator stories – aiming to get more precise, because we observed a lot of ambiguity lingering in there.
See the spreadsheet that @sasha has helped iterate on, including these top-level items:
Action (I want to [take an action]…
Goal (In order to… [receive a benefit])
validate a sample subset of my dataset
quickly check that my feed is compliant, without having to load my whole dataset
perform a minimal validation of all required fields in my schema
confirm that my data’s schema is minimally compliant with HSDS
perform a complete validation of all HSDS fields in my schema
confirm that my schema complies with the entire HSDS specification
validate my entire entire dataset for formatting compliance
confirm that my dataset complies with the entire HSDS schema and formatting conventions
validate my dataset for compliance with an HSDS profile
confirm that my dataset meets the precise specifications for my subdomain / context.
Test my endpoint for valid parameters
confirm that my API enables filters as specified in HSDS
perform a full validation of my API endpoint
ensure my API feed is publishing data that is compliant with the HSDS API.
to receive a JSON schema that specifies pass/fail per-field
be alerted to fields that need to be reformatted in order to be compliant
to receive machine-readable feedback that includes details about which fields have failed and why
understand what i need to fix in my schema (or dataset?) in order to be compliant.
receive human-readable feedback that includes details about which fields have failed and why
understand – without being a code person – what i need to fix in my schema (or dataset?) in order to be compliant.
(Note that the spreadsheet is a bit awkward in that some stories are duplicated across the different types of users – we’ll clean it up, would welcome suggestions.)
Our next steps will be to 1) confirm the wording for all of the above, then 2) confirm which of these are done by already-existing or in-development tools, then 3) rank the features that have yet to be built according to a) value and b) difficulty.
It’s not clear which apply just to API endpoints (aka web methods), which to HSDS structured data made available in a different way and which to both. I’m only interested in API compliance but people wanting to do offline tests might want different approaches to some points such as getting feedback
there’s the case of testing a few key API endpoints (web methods) (e.g. root, /services and /services/{id} which is separate from a full API test and separate from testing parameters. I don’t think this is in the list
Thanks, Mike. We can talk this through more on the call, but feel free to suggest more specific stories.
If I’m understanding this correctly, you’re saying there might be a distinction to make between validating parameters vs validating methods?
I observed the ambiguity of structured data vs API endpoints, but (forgive my lack of technical acumen here) isn’t validation of a JSON schema file also essentially like validating an API endpoint? Feel free to clarify here if we want to do another round of disambiguation between validating a JSON schema file vs API.
ensure my API feed is publishing data that is compliant with the HSDS API.
to receive a JSON schema that specifies pass/fail per-field
be alerted to fields that need to be reformatted in order to be compliant
to receive machine-readable feedback that includes details about which fields have failed and why
understand what i need to fix in my schema (or dataset?) in order to be compliant.
receive human-readable feedback that includes details about which fields have failed and why
These are the most important first steps from my point of view. There is some overlap here, but my main use case would be to send integration partners to the validator (and they will send me there too!) to confirm whether APIs are in-spec and available to use with our existing connectors/parsers. If not, we want helpful feedback as to what needs to be mended in order to work together.
I have added a few comments