OK notes from this conversation below and here, please feel free to help clarify / add / organize. This is a bit circular but a good discussion. we have a couple of specific questions we could pick up next. what do you want to discuss next?
Incentives –
“Local stewards are more likely to have relationships and are able to get people on the phone at the organization. More experienced data managers, however, understand the needs for the funders and how to craft a highly specialized and high quality record.”
Kate: “reconciling these two strengths is both necessary and possible.” experienced data managers can pass down their knowledge / build up data quality, while locals can build and maintain relationships.
“Duplicate record management is not always bad, in-fact some overlap in management of a singular resource drives more rich information and more frequent updates. Reconciling those records to match (suggest things are the same) and then map (human editing and confirmation of the matching)…” Creating a golden record set that distills this (?)
Who should get paid for what?
- Locals / domain experts/stewards for managing records on a per-record basis.
- Regional/state-level for bottom-lining comprehensive data, quality assurance, etc.
- Producing “opinionated” data sets for specific
- Participation in cooperative processes i.e. governance activities
- CBOs should get paid for providing services! This is important; it may or may not be relevant.
First two bullets can be gamed / played unequally. Organizations (local OR high-level/utility) might be incentivized to maximize their benefits in ways that aren’t in the best interests of users or the network as a whole. Example of incentivizing the wrong things: paying more for complex records leads to needless complexity.
^ but these challenges can be addressed through governance.Contract based payments with incredibly specific restrictions around what records are being compensated has really been proven to be effective in disincentivizing cheating in the system. Agreements on fair terms, and monitoring and conflict resolution processes to manage and evolve the agreements over time. These can also yield technical safeguards, though that has to follow agreements. “Tight standards and inclusion policies, especially coupled with auditing, make it so that each steward can keep their own records, that’s their business, and whether they get paid or not is on the basis of standards.”
Get clear on what we are trying to incentivize?
- Reliability.
- Responsiveness
- Relationships.
Are there different thresholds – some kinds of data are objective, can be collected with relatively minor levels of expertise; some kinds are specialized and need expertise.
Skyler: “Paying a per record fee when we are at a very early stage can be tricky. Knowing the actual cost of production of a record and having that be itemized is critical.”
Kate: “We need to find a way to recognize the value of checking a record even when significant information does not churn. We need to investigate measuring how an entire record is checked.”
Sasha: Time is tricky. a lot of the work will not be reflected in the metadata captured in digital tools. “Sometimes you need to chase an organization for 2 months, occasionally an organization will be really on top of it and hand the details to you proactively” “Phone conversations are currently offline and aren’t always able to be traditionally programmatically tracked. Beyond that, if you get sucked into a 50 minute conversation about a random side tangent with one organization” (David side thought: it helps build rapport and may help them be more helpful in the future so this is not a bad thing)
Kate: re expecting people to meet standards to participate, there is a level of expertise that involves personnel investment and specialization that can be difficult where the role is part-time or volunteer or not a primary responsibility.
Chris: “Being able to find the money in the system to support even just one FTE at a local level can really be differential for the quality of the data in that locality. Being able to earmark that funding and then allowing the local organization to get creative with how they spend that to drive results, can be a useful blend”
Can there be thresholds for compensation,
- Thresholds for simple vs complex
- Thresholds for minimal compliance with standards vs full compliance
We do want to track costs. But is it worth tracking all the metadata in order to compensate on a granular level?
Skyler:
1. Incentives are connected to Responsibilities, and
- We don’t assume that data driven metrics are the exclusive method for establishing compensation
Also, I’m personally thinking in terms of ratios of payment, not necessarily cost of record maintenance. I’m inclined to decouple those concepts for at least.
Kate –
Assuming we get funding for data maintenance:
A funder especially gov don’t want to pay a bunch of people. They want to fund one. So it naturally falls to a model where they fund a top-level organization, and the top-level picks the next level, and those pick their local partners. And funding flows differently across levels.
What are non-monetary incentives? Often at the lowest levels, the exchange might need to be other than money: training, software, access to data + analytics, support resources… what else?
Need roles for Auditing / review / standards-setting. This makes it hard to pass on top-level funding to lowest level, cuz it eats a lot up.
Who should pay for what?
There might be different kinds of payers in the market –
Funders that just want to pay for production of data (government?)
Funders that want to pay for specialized products – curation
We want proper staff… and we also want those local relationships that may be more informal.
So – Utility concept of bottom-line / overarching responsibility to aggregate and publish
Steward concept for designated responsibility that adheres to standards.
Community partner concept for leveraging local relationships.
A utility plays stewardship roles as well as administration, auditing, and curation.
A community partner may play a stewardship role, if they have the capacity / interest in assuming a higher level of responsibilities.
Should the concept of a ‘curator’ be distinct from (even if co-assigned with) ‘steward’? What about ‘auditor’?
Can / should we try to unbundle stewardship responsibilities for complex organizations, i.e. to distribute responsibilities across the service / location level?
David: “One thing that I think will be significant to investigate for capturing the value of the administration, education, and licensing is the travel agent and travel agency business model, and how those organizations interact with the major airlines, hotels, cruise lines, etc. Its a random spot of knowledge I have a decent amount of detail on, and the structure is really relevant to what I have learned”
