Validation - absent fields, nulls and empty strings

When using the validator against Open Referral UK version 3, we reject/fail empty strings where the field format is checked. This applies to:

  • service.url
  • service.email
  • organization.parent_organization_id

and probably elsewhere. This has caused a problem for one publisher that is sending empty strings.

We want to confirm what the correct API response should be for optional fields with no value. Options are:

  1. Fields with no data aren’t returned
  2. Fields with no data are returned as null
  3. Fields with no data are returned as ""

Without changes to the current validator options 2 & 3 will be issues when a field has a prescribed format e.g. uri , email

Hi Mike,

The short answer to the question is: Option 1; the publisher should not send empty strings and should omit fields where there is no data. Even for non-required fields. Your validator is behaving as expected.

This is — as you pointed out — that certain fields should be formatted a particular way e.g. a date, or a URI etc.

The desired behaviour for fields which the publisher cannot provide is to omit them. This is why they’re optional. Empty strings confuse the semantics of whether a particular field is actually included or not, shifting the burden to consumers of the data rather than producers. In some cases, this has knock-on effects leading to bad data quality and causing problems when someone tries to use the data in analysis or for rendering an interface.

1 Like

Thanks very much @mrshll. So the rule is “Omit the field (both label and value) if it is optional and you have no value.”

I understand that you will look to ensuring that this is covered in the HSDS guidance. I’ll raise the issue of clarification with the Open Referral UK team and advise the publisher who raised the issue.

1 Like

So the rule is “Omit the field (both label and value) if it is optional and you have no value.”

Spot on!

I understand that you will look to ensuring that this is covered in the HSDS guidance

Turns out that this is already covered!

Taken from the Mapping Data to HSDS:

  • For a given record, if there is no value for a particular property then that property should be omitted in the published JSON data unless the field is Required by the standard (see Schema Reference). It’s preferable to omit a property rather than include an empty value or null in JSON. For Tabular Data Package serializations, properties may have a null value.

I think this is sufficiently clear, but let me know if it needs adding elsewhere in the docs.