Skip to content

Conversation

@alilleybrinker
Copy link
Contributor

@alilleybrinker alilleybrinker commented May 5, 2025

This introduces a new Request for Discussion (RFD) process for considering changes to the CVE Record Format. The details of the proposal are in the commit.

This commit includes both the first RFD, proposing the introduction an RFD process and modeling what the process looks like, as well as a template for future RFDs to reuse.

EDIT:

@alilleybrinker alilleybrinker force-pushed the alilleybrinker/rfd-process branch from 8f91091 to e3176dc Compare May 5, 2025 20:46
@alilleybrinker
Copy link
Contributor Author

alilleybrinker commented May 5, 2025

First comment to track any open points of disagreement as they arise...

(Checked off items have been resolved via updates to the PR materials)

  • Amend link to purchase Seeing Like a State to an alternative that doesn't require payment.
  • Clarify language around publishing the results of CVE Board votes arising from RFD proposals in the QWG.
  • Clarify language around timelines for RFD consideration and auto-closure of inactive RFDs.
  • Clarify procedures for accepting RFD pull requests
  • Should the RFD text mention SchemaVer?
    • Resolved: No, while many in the QWG have raised SchemaVer as a candidate model for versioning the CVE Record Format, it is not formally adopted, and there are reasons to consider other models. There should be a separate RFD opened to resolve questions around versioning the CVE Record Format.
  • Should CVE commit to forward compatibility guarantees between versions of the schema?
    • Resolved: This does not need to be resolved in the current RFD; the text of the RFD has been updated to reflect a need for RFD submitters to address both backward and forward compatibility concerns. A more complete resolution of precise forward compatibility guarantees for the schema should be achieved through a separate RFD on versioning the CVE Record Format.
  • Are the terms "backward compatibility" and "forward compatibility" acceptable?
    • Resolved: Yes, both are widely used terms with well-understood definitions, and the RFD text provides definitions for both.
  • Are the definitions for "backward compatibility" and "forward compatibility" in the RFD text sufficiently clear?
    • Resolved: Yes, while there may be some debate about the definitions of the terms in the abstract, both are given definitions in the RFD text. There is a broader question about the rigor of the versioning rules around the CVE record format which may in the future prompt a rewrite of the "Compatibility and Migration" section, but which does not need to be resolved for this proposal to be accepted.

@darakian
Copy link
Contributor

darakian commented May 6, 2025

I love this. Thank you for putting the effort in! For the benefit of of anyone else trying to read the full document here's a link to see it rendered.
https://github.com/CVEProject/cve-schema/blob/e3176dc1fb18aee25e96c3fb3987895b5fdec25b/rfds/0000-establishing-the-rfd-process.md

Some nits:
Linking out to a paid (albeit cheap) book https://yalebooks.yale.edu/book/9780300078152/seeing-like-a-state/ limits accessibility. Please also include something like a google books link which will help readers find a free local library copy
https://www.google.com/books/edition/Seeing_Like_a_State/Qe_RDwAAQBAJ?hl=en&gbpv=0

Final approval of all proposals for the CVE Record Format must be done by the CVE Board. While the QWG has the power to refuse to advance a proposal to the Board if they do not consider it worthwhile to pursue, the Board makes the decision to approve or reject a proposal after the QWG advances it to them.

For the sake of procedure; does this mean that the CVE board should record their approval via PRs? eg. One/many board member(s) approve/reject a PR?

A (minor) disagreement:
I'd like to suggest that there should be strictly enforced timeline/timeliness on RFDs where some automated action closes RFDs which have been inactive for three months. This would require that those proposing changes stay engaged in the process. Something like the stale bot which tends to pull requests on the github database could be made.

example: github/advisory-database#5234
tl;dr: a bot looks at PRs once a day, warns users on the PR after some period and adds a stale tag. Comes back later and auto closes anything without activity.
source: https://github.com/github/advisory-database/blob/main/.github/workflows/stale.yaml

@alilleybrinker
Copy link
Contributor Author

Thanks! I'll update the reference link for "Seeing Like a State." It's really only included for the purpose of explaining the term "legible," which is covered in the Wikipedia page for the book, so I'll link to that.

I can also amend the first post of the PR to include rendered links to the RFD and the template.

For final approval, I'm inclined to make it the job of a QWG Co-Chair to update the PR with the result of a CVE Board vote, since this RFD process is something the QWG is adopting for itself, and thus ought not to bind / demand new process from anyone outside of the QWG.

I'm also happy to amend with an activity requirement to stave off automated closure, assuming someone in the QWG is willing to set up that automation as you describe.

@alilleybrinker
Copy link
Contributor Author

Ah, actually @darakian I think the current language matches what you want pretty well. There's no set requirement for the QWG to resolve an RFD with a final determination on a set timetable, but there is a requirement that RFD pull requests remain active, with inactive PRs being closed and the RFD then requiring re-submittal.

I'll change the closure language from "may be closed" to "shall be closed" to make it non-discretionary on the part of the QWG Co-Chairs.

@alilleybrinker alilleybrinker force-pushed the alilleybrinker/rfd-process branch from e3176dc to 51fd2dc Compare May 7, 2025 16:38
@alilleybrinker alilleybrinker changed the title Introduce an RFD process. RFD to introduce an RFD process May 7, 2025
@darakian
Copy link
Contributor

darakian commented May 8, 2025

I'll update the reference link for "Seeing Like a State." It's really only included for the purpose of explaining the term "legible," which is covered in the Wikipedia page for the book, so I'll link to that.

Wikipedia is also good and my nit there is of course minor. I just want to be mindful of open access principles as much as possible.

For final approval, I'm inclined to make it the job of a QWG Co-Chair to update the PR with the result of a CVE Board vote, since this RFD process is something the QWG is adopting for itself, and thus ought not to bind / demand new process from anyone outside of the QWG.

That seems reasonable to me as well, but I do think the board should make it clear that they accept that delegation of responsibility (maybe via RFD even 🤔).

I'll change the closure language from "may be closed" to "shall be closed" to make it non-discretionary on the part of the QWG Co-Chairs.

That's fair too. The automation aspect is mostly just a forcing function to keep things moving forward.

axeberg
axeberg previously requested changes May 8, 2025
Copy link

@axeberg axeberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo in list of items.

@alilleybrinker alilleybrinker force-pushed the alilleybrinker/rfd-process branch from 51fd2dc to a53be9e Compare May 8, 2025 16:23
@ElectricNroff
Copy link

is the anchor text in the Value for Field RFD Pull Request always

RFD #0000

or is it supposed to be a unique identifier? I'm guessing (but I'm not sure) that

https://github.com/CVEProject/cve-schema/blob/a53be9e24643b313b56be85912d14992dfd398a1/rfds/0000-establishing-the-rfd-process.md

and

https://github.com/CVEProject/cve-schema/blob/c1b0d8e682186c46cc9d1c97ebb5178b58e872b9/rfds/0000-expanding-software-ids.md

are supposed to have different anchor text such as

RFD #0000

versus

RFD #0001

and maybe the .md filenames are not both supposed to start with 0000-.

@alilleybrinker
Copy link
Contributor Author

alilleybrinker commented May 8, 2025

Per the RFD procedures specified in the RFD document, they only get assigned a number if/when they are approved. Until that point they use 0000 as a placeholder.

On approval then yes the document file name and contents will be updated to reflect the number assigned to the RFD.

@alilleybrinker
Copy link
Contributor Author

I have now opened a poll on whether to adopt this RFD: #412

@boblord
Copy link

boblord commented May 22, 2025

This introduces a new Request for Discussion (RFD) process for considering changes to the CVE Record Format. The details of the proposal are in the commit.

Great work. Here are my suggested edits for the template:

  1. For the problem statement, add a requirement to articulate the severity of the problem today, and who it affects. We all have our pet peeves and we need to prioritize those problems that are causing the most harm, or impeding the most progress.
  2. Also for the problem statement, add "What happens if we do nothing?" I've found that this question is incredibly clarifying.
  3. For the proposed solution, add a requirement to list other possible "correct solutions" and why this specific one is the best for the identified stakeholders and use cases.
  4. Recommend being more explicit in the metrics system. This is a muscle we have not exercised in the past, so being more verbose in this document will help proposal writers.

This introduces a new Request for Discussion (RFD) process for
considering changes to the CVE Record Format. The details of
the proposal are in the commit.

This commit includes both the first RFD, proposing the
introduction an RFD process and modeling what the process
looks like, as well as a template for future RFDs to reuse.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@alilleybrinker alilleybrinker force-pushed the alilleybrinker/rfd-process branch from a53be9e to 7de8fbb Compare May 22, 2025 18:19
@alilleybrinker
Copy link
Contributor Author

Thanks @boblord! I've amended the template with your suggestions. Regarding alternative options, I opted not to modify the "Proposed Solution" section, as alternative options are already given space to be listed in the "Related Issues or Proposals" section of the template.

@darakian
Copy link
Contributor

@alilleybrinker one meta comment I have is that you should avoid all the squashing and force pushing of commits. It has the effect of deleting the history of how the final form of the document comes together. Some commits are worth squashing together and this is an art form more than a science, but I would encourage you to err more on the side of leaving more breadcrumbs rather than fewer 👍

@alilleybrinker
Copy link
Contributor Author

alilleybrinker commented May 22, 2025

@darakian yeah this is a frustration I have with how GitHub pull requests handle code review. In other systems, comments against prior versions of the patchset included in a change are much clearer to navigate and review. In GitHub, the history isn't lost (if you scroll up, the force-pushed-over commits are still accessible and show prior revisions of the documents), but they are inconvenient to access.

In general, I prefer to keep the commits in a PR in a merge-able state, rather than either a squash-at-the-end approach immediately prior to merge or merging all intermediate commits.

That said, if the group's preference is to merge intermediate commits, or to squash at the end, I'm happy to adjust. This may be a good thing to document in a CONTRIBUTING.md.

EDIT: I'll do additional commits going forward, and have now added one to address the feedback from @andrewpollock.

Copy link
Contributor

@andrewpollock andrewpollock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fantastic and revolutionary leap forward in structural discipline, thank you!

This clarifies the procedures around updating the RFD number
in an RFD pull request after approval of an RFD by the
CVE Board.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@ElectricNroff
Copy link

I feel that SchemaVer has only a small value in characterizing the change proposed in an RFD, and that additional characterization of the change is needed within an RFD. In other words, I don't agree that "describe any migration paths" is an adequate alternative.

SchemaVer is only about interaction between a new schema and historical data. Sometimes, it only represents the amount of work that the CVE Program administration would need to perform in order to convert historical data to the new format. Often, it does not capture the amount of work that downstream consumers would need to perform for continuity of operations.

I don't know of previous research into measuring downstream impacts of JSON schema changes that alter the contract between producers and consumers. This type of research may exist. I believe that relevant yes/no-style characterization includes:

  • does the new schema extend a closed set of options (i.e., a composition or an enum)?

  • is that closed set mutually exclusive?

  • is that closed set found within required properties?

  • is that closed set found within data elements that are known to be very important to consumers?

  • does a new data element require consumers to have a new secondary parser (for continuity of operations)?

  • is the secondary parsing complex enough that consumers may reasonably need to audit a secondary parser implementation before using it?

Depending on how the schema change is characterized, it may be easier to make decisions on:

  • how long should the schema change be announced on a CVE Program public roadmap before going into production?

  • what types of testing are needed before the schema change goes into production?

  • how far should the schema version number be advanced to capture the magnitude of the change, e.g., from version 5 to version 6?

  • how much consumer demand for the schema change is needed before it is initially accepted for a public roadmap?

There once was work on the SOAP specification that I think is tangentially related: https://w3.org/TR/soap12-part1 sections 2.4 and 5.2.3. Essentially, SOAP has the concept of mustUnderstand. This cannot be directly applied to a JSON schema, but parts of the analogy might help to illustrate the above points.

Today, state in a CVE Record must be in the enum-based closed set of PUBLISHED or REJECTED. Suppose this were changed to allow PARTIALLY_PUBLISHED to support the https://www.cve.org/ResourcesSupport/Glossary concept of "non-trivial information about a vulnerability is publicly available." This would be roughly the same as asserting mustUnderstand: true for PARTIALLY_PUBLISHED. In essence, this is a substantial schema change because a consumer cannot ingest PARTIALLY_PUBLISHED records as if they were PUBLISHED records, i.e., a PARTIALLY_PUBLISHED record is only guaranteed to have one non-trivial data element, and not all of the required data elements for the PUBLISHED state.

One might imagine similar use of mustUnderstand when a new schema introduces additional valid choices to a oneOf-based closed set.

By contrast, suppose that CVSS 3.1 were the only choice in metrics and then a new schema introduced CVSS 4.0. However, the metrics property, like today, is not required. Here, mustUnderstand: true would not be applicable. A consumer may start to see records with CVSS 4.0 (and no CVSS 3.1), but they are no worse off than if the producer simply offered no metrics at all. The producer is not omitting any previously required information, nor are they moving any previously required information to a different location in the document.

Again, mustUnderstand is simply a connection to previous work. I am not suggesting that every concern is addressed if an RFD header includes a mustUnderstand field.

@alilleybrinker
Copy link
Contributor Author

@ElectricNroff, the template purposefully leaves the level of required detail on a migration path open to the discretion of the QWG. Some proposals are substantial and will, as you describe, entail a more substantial migration requiring detailed planning and public communication. Other changes may be minor and require little work. I don't want to be overly prescriptive about the required process here, because I think the odds of getting a one-size-fits-all process right up-front in a document like this is low.

We require outlining the level of SchemaVer impact because SchemaVer is what the project uses for versioning, and the migration path is intended to cover the remaining details (to the level necessary) to support an ecosystem migration on both the production and consumption sides to support a change.

At Matt Powers' recommendation, this amends the "Compatibility and
Migration" section of the RFD template to describe expectations for
analyzing and planning migrations more clearly, including specific
questions which ought to be answered, and clarifying the limits of
SchemaVer in expressing the adoption burden of new features.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@alilleybrinker
Copy link
Contributor Author

@ElectricNroff, I've amended the RFD's "Compatibility and Migration" section to hopefully address your concerns. Take a look and let me know if you think it still needs revision!

Based on conversations with Matt Power, I've amended the text
of the "Compatibility and Migration" section of the RFD template
in a couple of ways:

1. Removed explicit reference to SchemaVer as the versioning scheme
   of choice for the CVE Record Format.
2. Introduced a set of questions which must specifically be
   answered in any submitted RFD.

The key goal here is to balance the need for a high level of rigor
around compatibility and migration in CVE because it is a
systemically-important and large multi-stakeholder system with the
need to not be overly prescriptive in the RFD template in ways
which may in fact reduce rigor by over-specializing RFD requirements
on today's considerations for what the important questions are.

The questions included are specific, and intended to capture key
concerns about forward compatibility and the impact of changes
on CVE consumers particularly.

Separately, references to SchemaVer were removed because, while
it is my understanding that SchemaVer is the version scheme of
choice for the CVE Record Format, that is not currently codified
and itself likely ought to go through an RFD process to firmly
resolve. In fact, Matt Power has raised concerns that SchemaVer
may be insufficiently expressive for the versioning constraints
under which CVE operates, and I'd rather not attempt to resolve
that question in the context of this process-focused RFD.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@alilleybrinker
Copy link
Contributor Author

@ElectricNroff I've updated the Compatibility and Migration section based on our conversation yesterday. Please take a look and let me know if this satisfies what you're looking for. (CC @ccoffin)

@ElectricNroff
Copy link

I feel that the term "forward compatibility" is needlessly confusing because there are different vantage points with different compatibility concerns. For example, suppose that the CVE Record Format stated that vulnerabilities would be described in English in either an HTML document or a PDF document. Then someone proposed that a Word document would also be OK. That would be an "ADDITION" type of change, but would not be backwards compatible with the processes of consumers to read the documents.

@alilleybrinker
Copy link
Contributor Author

alilleybrinker commented Jun 5, 2025

I feel that the term "forward compatibility" is needlessly confusing because there are different vantage points with different compatibility concerns. For example, suppose that the CVE Record Format stated that vulnerabilities would be described in English in either an HTML document or a PDF document. Then someone proposed that a Word document would also be OK. That would be an "ADDITION" type of change, but would not be backwards compatible with the processes of consumers to read the documents.

I believe this is an incorrect use of the term "backward compatibility." Backward compatibility is when a new version of something is still compatible with prior instances of that thing. So adding a field (an ADDITION-level change in SchemaVer) is inherently backwards compatible, and thus the proposed change in your hypothetical is backwards compatible.

To be more succinct:

  • Backwards compatibility: When a new version of the schema accepts records compatible with the past version of the schema.
  • Forward compatibility: When a new version of the schema does not permit records which would not be permitted by the past version of the schema.

The one wrinkle in forward compatibility is that, if it's not already documented, we'd need to document what additions we expect CVE consumers to be resilient to, at minimum: adding new optional fields to an object.

@ElectricNroff
Copy link

In the hypothetical, a client could sent a GET request today to the CVE Services server for /api/cve/CVE-2026-0001 with the header "Accept: application/pdf, text/html" and they would get the CVE-2026-0001 document in either PDF or HTML (whichever the server chose to send). However, the CNA can choose to update the record to Word tomorrow if the proposal were adopted. Then, if I send the same GET request for /api/cve/CVE-2026-0001 with the "Accept: application/pdf, text/html" header, I receive an error message to the effect that the server does not have CVE-2026-0001 in either PDF or HTML. In this situation, it is completely valid to say that the API service is not backward compatible. (It could be made backward compatible if on-the-fly data conversion were implemented.)

That is what I meant by "different vantage points."

@darakian
Copy link
Contributor

darakian commented Jun 5, 2025

@ElectricNroff I don't follow with the pdf/word/etc... stuff, but the term forward compatibility is not some new invention. Wikipedia has an article with edits going back to at least 2003
https://en.wikipedia.org/w/index.php?title=Forward_compatibility&action=history&offset=&limit=500

A question for you. Say we have a schema of the form

{
field_a: String
}

A single named field that holds a string.
Would you consider the following schema to be backwards compatible or not?

{
field_a: String,
field_b: String
}

@ElectricNroff
Copy link

@ElectricNroff I don't follow with the pdf/word/etc... stuff, but the term forward compatibility is not some new invention. Wikipedia has an article with edits going back to at least 2003 https://en.wikipedia.org/w/index.php?title=Forward_compatibility&action=history&offset=&limit=500

A question for you. Say we have a schema of the form

{
field_a: String
}

A single named field that holds a string. Would you consider the following schema to be backwards compatible or not?

{
field_a: String,
field_b: String
}

In the way you've written it, no, because a producer might have already used field_b in an incompatible way. When the schema was:

{
field_a: String
}

a producer may have already composed a historical document:

{
  "field_a": "apple",
  "field_b": false
}

and this won't validate against the new schema. This is analogous to "we decide to add a new cost property" in SchemaVer.

Possibly you meant that "additionalProperties": false was present in the first schema. That normally changes the answer to yes.

In that situation, the most common JSON schema social contract is that ordinary consumers aren't allowed to raise compatibility objections for a schema change that only introduces extra data elements. (For example, the extra data has no effect on what data is required, the valid ways of expressing required data, what other data may be present, what values are allowed in other data, or the steps for extracting information from any data.) If they aren't interested in field_b, they are expected to ignore it.

There can be domain-specific differences in the social contract, e.g., if everyone using this schema produces multi-TB strings and understands that two of them aren't going to fit in memory on the users' machines, but I think these are uncommon.

@darakian
Copy link
Contributor

darakian commented Jun 6, 2025

In the way you've written it, no, because a producer might have already used field_b in an incompatible way.
...
Possibly you meant that "additionalProperties": false was present in the first schema. That normally changes the answer to yes.

Say "additionalProperties": false is not present, but we can be 100% sure that no records have ever been published using field_b. Do you consider that breaking? If so, how does one determine what keys are available for use? Is it only in those sections that additionalProperties is set to false?

@alilleybrinker
Copy link
Contributor Author

I don't think we need to resolve the versioning question here. For the purposes of the RFD template, the goal is to include enough detail on what should go into an RFD so that the QWG can be well-informed when making decisions. This template could also be updated in the future after an RFD on how to version the CVE Record Format if there's a need to reflect new consensus on the right questions.

I think the current text is probably sufficient at asking the right questions and probing for the right detail. I also think backwards compatible and forward compatible are sufficiently well-attested terms with common and well-understood meanings, and I am comfortable leaving them in the template.

@alilleybrinker
Copy link
Contributor Author

I've opened an issue on the need to develop a CVE Record Format versioning RFD, which links to this and two other open discussions that touch on the versioning question. As I said, I think this RFD can and should move forward without waiting to resolve the versioning questions raised in the linked issue. (#418)

@alilleybrinker
Copy link
Contributor Author

@ElectricNroff and @darakian, how do you both feel about the text of the RFD as it stands, with the understanding that we won't try to resolve the versioning commitment question here, and with the clarification that "forward compatibility" is a standard term with a well-understood definition?

I think, given those two notes, that the open issues are addressed, but I want to confirm.

@ElectricNroff
Copy link

I don't think we need to resolve the versioning question here. For the purposes of the RFD template, the goal is to include enough detail on what should go into an RFD so that the QWG can be well-informed when making decisions. This template could also be updated in the future after an RFD on how to version the CVE Record Format if there's a need to reflect new consensus on the right questions.

I think the current text is probably sufficient at asking the right questions and probing for the right detail. I also think backwards compatible and forward compatible are sufficiently well-attested terms with common and well-understood meanings, and I am comfortable leaving them in the template.

No, for APIs that depend on JSON schemas, the reality is that there are two different meanings of (for example) backward compatible that are both in common use, neither one is the result of any mistakes or misunderstandings, and they represent two different perspectives. For example:

https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html

BACKWARD compatibility means that consumers using the new schema can
read data produced with the last schema.

https://docs.specmatic.io/documentation/backward_compatibility_rules.html

Backward Compatibility Rules ... Responses ... Mandatory key becomes
optional ... incompatible ... It's possible that the server will not
send this key. Some consumers may be expecting it, but will not be
receiving it henceforth.

Suppose the old schema is:

{
  "type": "object",
  "properties": {
    "x": { "type": "string" },
    "y": { "type": "string" }
  },
  "required": ["x", "y"]
}

and the new schema is:

{
  "type": "object",
  "properties": {
    "x": { "type": "string" },
    "y": { "type": "string" }
  },
  "required": ["x"]
}

There is v1.0 of an API service that responds with documents that conform to the old schema. Then, the API service is changed to v1.1 and responds with documents that conform to the new schema.

In the confluent.io perspective, this is a backward compatible change because a client who is using the new schema can certainly read and understand data produced with the old schema (i.e., all documents are guaranteed to have both x and y).

In the specmatic.io perspective, this is not a backward compatible change, because a mandatory key y has now become optional. The server is now free to send documents that do not have y. Client code that previously was accessing the y key may now fail with an error message indicating that the key was not found, or the client may not be able to compute a y value that is needed for an important business purpose.

The confluent.io perspective is occasionally called schema-backward-compatible, whereas the specmatic.io perspective is occasionally called consumer-backward-compatible. (Normally, they are both just called backward compatible, perpetuating the confusion.)

They are both of interest, but they affect the CVE Program in different ways. We are concerned about the consumer experience. We don't have evidence of any significant consumer population that retrieves JSON documents (either from our API service, or from a known filename on GitHub) and then executes JSON schema validation. We do know that consumers access JSON keys that they expect are present, and use the values in important decision processes of varying complexity.

So, I feel that it's best to offer explanations that apply both to the consumer audience and the RFD author audience. Regardless of what terminology we choose, we have to explain what we mean: we cannot rely on an assumption that something might be "well-attested" one way or the other. For example, some relatively simple initial statements could be:

When moving from CVE Record Format x.y to x.(y+1), should the change be schema-backward-compatible?
Historically, no. We have made changes that forbid documents that we considered to be using a loophole, e.g., "additionalProperties": false was accidentally missing somewhere. To preserve the consumer experience, the CVE Program administrators arrange to update the server content (and consequently the data copy on GitHub) so that all CVE Records conform to the stricter schema. However, thus far, we have not made a change to block types of document structures that we feel were being legitimately used, when that change is not schema-backward-compatible.
When moving from CVE Record Format x.y to x.(y+1), should the change be consumer-backward-compatible?
Almost certainly yes. It is especially problematic when required or popular data elements would be affected by a change that is not consumer-backward-compatible.
When moving from CVE Record Format x.y to (x+1).0, should the change be consumer-backward-compatible?
This is not necessary, as long as the community experience is substantially improved by the change that is not consumer-backward-compatible.

@alilleybrinker
Copy link
Contributor Author

@ElectricNroff I appreciate you digging into different examples of how others use those terms. That said, the RFD provides definitions for both "backward compatibility" and "forward compatibility."

@ElectricNroff
Copy link

@ElectricNroff I appreciate you digging into different examples of how others use those terms. That said, the RFD provides definitions for both "backward compatibility" and "forward compatibility."

I believe that there would be substantial challenges in using its definitions:

To address backward compatibility, explain if and how your
proposal would impact users of the schema's ability to parse
existing CVE records produced under prior versions of the CVE
Record Format.

To address forward compatibility, explain if current users of
the schema would be able to accept all, some, or none of the
records produced with the schema as modified by your proposal.

Separately, you wrote:

#405 (comment)

Backwards compatibility: When a new version of the schema accepts
records compatible with the past version of the schema.

Forward compatibility: When a new version of the schema does not
permit records which would not be permitted by the past version of
the schema.

(shown here just as background information - it's not part of the RFD document)

To restate what I've said in a different way:

"impact users of the schema's ability to parse existing CVE records produced under prior versions of the CVE Record Format" does not mention that, within CVE Program operations, these users are not expected to try to parse "records produced under prior versions of the CVE Record Format." When the schema changes, the official copies of all CVE Records are changed (e.g., administratively) to comply with the current version of the CVE Record Format.

"explain if current users of the schema would be able to accept" does not indicate what "accept" means. Typically, it is easy for users to download the latest copy of the schema, but not necessarily easy for them to change how they extract and use data within the now validated documents, and not necessarily easy for them to change a UI so that (for example) users can enter any now valid data but are dynamically guided away from entry of invalid data.

For "When a new version of the schema accepts records compatible with the past version of the schema": a previous update to the CVE Record Format (5.0.0 to 5.1.0) was widely favored, and had zero known user complaints afterward, even though this property was not present.

For "When a new version of the schema does not permit records which would not be permitted by the past version of the schema": a previous update to the CVE Record Format (5.1.0 to 5.1.1) was mostly favored, and had zero known user complaints afterward, even though this property was not present.

By contrast, the material at https://docs.specmatic.io/documentation/backward_compatibility_rules.html offers objective criteria about what disrupts a consumer's compatibility experience in realistic scenarios. (It also offers automation around this.) It may not all be accurate, and it may not be complete, and it may not be adapted to our exact use case. But, my guess is that it would allow evaluation of schema-change ideas with weeks or months less elapsed time.

@alilleybrinker
Copy link
Contributor Author

@ElectricNroff I've updated my tracking of open issues to reflect some of the recent things we've covered and where I believe the conversation stands: #405 (comment)

The current question is whether the RFD's definitions of "backward compatibility" and "forward compatibility" are sufficiently clear.

Based on your latest comment, I will amend the language slightly, but I will not produce a detailed breakdown of the precise versioning guarantees like is found in the linked Specmatic documentation, because I do not believe I can.

The QWG has not resolved the rules for CVE Record Format versioning to that level of detail. In fact, we only just finished a discussion earlier in this thread objecting to SchemaVer, which was previously the best candidate (and indeed, which I had understood to be the group's consensus) model for versioning the Record Format.

I understand your desire for more rigor in this section, and I agree with the goal and want to see the QWG get there. I think for now the best we can do is to have general requirements for describing compatibility in the RFD template, with an understanding that the QWG will need to closely scrutinize this section and ask pertinent questions during review.

At the same time, we can pursue an RFD to codify precise versioning guarantees (see: #418), and when that's resolved we can amend the RFD template to be more detailed.

Amend the definitions of "backward compatibility" and
"forward compatibility" in the "Compatibility and Migration"
section of the RFD template to more clearly explain
requirements for RFD writers.

In the future we'd like this section to be more rigorous,
and ideally to provide a detailed breakdown of what kinds
of changes in the schema are considered "breaking," but
the versioning rules for the CVE Record Format aren't
sufficiently defined yet.

I recommend developing an RFD to solidify the versioning
rules, and then amending the template to require greater
rigor around compatibility in this section.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@boblord
Copy link

boblord commented Jun 12, 2025 via email

@alilleybrinker
Copy link
Contributor Author

Which downstream consumers of CVE data have we polled on this topic? Should we bring more into the conversation?

I think consumer engagement would make sense for the versioning conversation, but for this RFD which is solely focused on the process the QWG follows for change management I don't think it makes sense.

@alilleybrinker
Copy link
Contributor Author

Note

Final Comment Period called!

This proposal is entering a Final Comment Period (FCP). There is one week remaining to give comments on this proposal, closing at 1pm PDT / 4pm EDT, Thursday, June 19th.

@alilleybrinker
Copy link
Contributor Author

Note

Final Comment Period Closed

The Final Comment Period (FCP) for this proposal has closed, with no new issues raised. This proposal is now closed for further issues, and goes to the QWG Co-Chairs to determine whether to adopt or reject the proposal.

As this is a QWG-specific procedural proposal, the QWG Co-Chairs will be the final deciders, and the issue will not advance to the CVE Board.

@alilleybrinker
Copy link
Contributor Author

Update from the last QWG meeting, held yesterday. With the FCP closed, @ccoffin has brought the RFD to the attention of the CVE Board for any comments. As the RFD only proposes procedural changes for the QWG itself, it is not necessary for the Board of directors to vote to approve it, as they would for changes to the CVE Record Format. However, it is being offered to the Board for comment, as it is a substantive change that impacts the nature and format of proposals which will in the future be presented to the Board after QWG approval.

Barring the raising of additional concerns by members of the CVE Board, this RFD (and thereby the RFD process itself) will be adopted by the QWG.

@alilleybrinker
Copy link
Contributor Author

Update from the 2025/07/03 QWG meeting: the RFD will be adopted, as we have not received any contrary feedback from the CVE Board. Next steps will be:

  • Amend this PR to assign the RFD number 1
  • Merge the PR

@ccoffin, I'll update the PR with the RFD number and ping you when it's ready to merge.

Per the RFD rules, assign ID CVEProject#1 to the first RFD.

Signed-off-by: Andrew Lilley Brinker <[email protected]>
@alilleybrinker
Copy link
Contributor Author

@ccoffin, the RFD ID number has been updated. This is ready for merge!

@ccoffin ccoffin dismissed axeberg’s stale review July 7, 2025 17:09

no need for this review

@ccoffin ccoffin merged commit 27044cd into CVEProject:develop Jul 7, 2025
1 check passed
@alilleybrinker alilleybrinker deleted the alilleybrinker/rfd-process branch July 25, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants