Skip to content

Conversation

michaelehab
Copy link
Collaborator

@michaelehab michaelehab commented Jul 25, 2025

Solves #1953 and #1902

  • Add PyPa live pipeline importer to fetch advisories affecting a single PURL
  • Add tests for PyPa live importer to test different scenarios and mock the GitHub API
  • Add a new API endpoint to run live evaluation importers
  • Add tests for the live evaluation API endpoint
image image

* Add PyPa live pipeline importer to fetch advisories affecting a single PURL

* Add tests for PyPa live importer

Signed-off-by: Michael Ehab Mikhail <[email protected]>
* Add a new API endpoint to run live evaluation importers

* Add tests for the live evaluation API endpoint

Signed-off-by: Michael Ehab Mikhail <[email protected]>
@michaelehab michaelehab changed the title Add PyPa live pipeline importer Add Live Evaluation API endpoint and PyPa live pipeline importer Jul 30, 2025
@michaelehab michaelehab requested a review from TG1999 July 30, 2025 13:00
michaelehab added a commit that referenced this pull request Aug 18, 2025
* Add GitHub OSV Live V2 Importer

* Add tests for the GitHub OSV Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 18, 2025
* Add Curl Live V2 Importer

* Add tests for the Curl Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 18, 2025
* Add Gitlab Live V2 Importer

* Add tests for the Gitlab Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 18, 2025
* Add NPM Live V2 Importer

* Add tests for the NPM Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 21, 2025
* Add Elixir Security Live V2 Importer

* Add tests for the Elixir Security Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 21, 2025
* Add Postgres Live V2 Importer

* Add tests for the Postgres Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab added a commit that referenced this pull request Aug 21, 2025
* Add PySec Live V2 Importer

* Add tests for the PySec Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <[email protected]>
Signed-off-by: Michael Ehab Mikhail <[email protected]>
@michaelehab
Copy link
Collaborator Author

@keshav-space @TG1999 I worked on modifying the Live Evaluation API based on recent discussions. Currently, the live evaluation API enqueues jobs to the "live" rq, when we have workers listening to it, the live evaluation becomes async.
image
I also added a LivePipelineRun model to help group live pipeline runs for a given live evaluation submission. We return the live run id in the live evaluation API and the user can use this live run id to check the current state of the given live evaluation request, seeing even details about which live importers ran and their current state.
image

Copy link
Member

@keshav-space keshav-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michaelehab , see a few suggestions. Also this needs an update in our docker compose for new live evaluation worker.

Signed-off-by: Michael Ehab Mikhail <[email protected]>
Signed-off-by: Michael Ehab Mikhail <[email protected]>
Signed-off-by: Michael Ehab Mikhail <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants