Reposted : Add Support for Offline Catalog Files in OSD (Feature/support_offline_catalog)

[KatsuhiroWatanabe]

Pull Request: Add Support for Offline Catalog Files in OSD Software

This update introduces support for using offline catalog files in the OSD environment, enhancing flexibility for scenarios where the default OSD module cache may not be suitable.

Background

Since mid-April, daily OSD module updates have included OS and DriverPack catalog files (JSON and XML) under the cache folder. These updates ensure access to the latest download information.

However, some users may require customized or alternative catalog files. To support this, the update allows placing user-defined catalog files in the USB\OSDCloud\Catalogs folder.

Key Features

• Offline Catalog Support: Users can now override the default cache by placing their own catalog files in the Catalogs folder.
• URL Validation: To prevent issues from outdated or invalid URLs in custom catalogs, a curl-based validation step has been added. This has proven effective in internal testing.
• Non-Intrusive: This change does not affect normal operations unless the Catalogs folder is explicitly created.

Request

Please consider merging this enhancement into the master branch of the OSD project.

EDIT (7/21/2025)

feature/InintStartNet2 (PR:#288), cherry-picked for integration:

[skyblaster]

This looks excellent, and since you're only excluding C: from the search path, it should work with network boot scenarios where no USB drive is present (ie. embedded in the boot.wim at X:\OSDCloud\Catalogs).

I would love to be able to define custom paths such as the following, and then leverage the "not yet implemented" cache host variable (see #272), which looks like something we could do with this patch:
"Url": "{HOSTNAME}/OSDCloud/OS/24H2-latest.wim"

[KatsuhiroWatanabe]

Thank you for the feedback!

Originally, since the current OSD reloads catalog files from the cache under the module path, we wanted an option to use only company-validated catalogs (Drivers and ESDs) stored in a different location, rather than relying solely on the default module path.

While the OSD module regularly updates cached catalog files via GitHub Actions—allowing us to always use the latest versions—there is currently no way to specify and use a particular version of a catalog file.

That’s why I proposed this pull request: to optionally allow importing catalog files from drives (excluding C:) under the \OSDCloud\Catalogs path.

As you pointed out, this also enables scenarios where the catalog is embedded in the boot.wim (e.g., at X:\OSDCloud\Catalogs) or stored on USB devices. The object list will be generated from the last successfully loaded catalog file, so you can take advantage of this behavior if it fits your use case.

Our ultimate goal is to host validated catalog files either on USB or online, and always use the latest version from those sources.

I’m also planning to propose another pull request to:
Add an option to disable automatic updates of the OSD module (to prevent overwriting custom code), and
Support optional PowerShell script execution after network connection (e.g., to download company-maintained catalogs from the internet).

We don’t necessarily expect this pull request to be merged as-is, since we are already operating with a patched custom version internally. However, we would greatly appreciate it if equivalent functionality could be considered for inclusion in the official OSD module.

[skyblaster]

Hi @KatsuhiroWatanabe, do you happen to have an example user defined OS catalog you could share?
Have you tested it with the new GUI in OSD.Workspace invoked by Start-OSDCloudWorkflow or only with Start-OSDCloudGUI?

EDIT: Sorry for the dumb question. I see this PR is geared towards OSDCloud v1.
The new functions appear to be Get-OSDCatalogOperatingSystems and Get-OSDCatalogDriverPacks

[KatsuhiroWatanabe]

Hi @skyblaster,

Thanks for your comment!

Yes, I can share an example of a user-defined OS catalog. It's a very simple one — just a modified version of the original JSON (cache\archive-cloudoperatingsystems\cache\archive-cloudoperatingsystems) with updated values for the OS download URL and SHA1 hash. Since it's quite straightforward, I didn’t include it here, but I’d be happy to upload it if needed.
(In our case, we had to patch the ja-jp.esd, so we tweaked the JSON and uploaded the patched ESD to our Azure Blob. At this moment, the issue is resolved by using Config/Scripts/Startup/Shutdown scripts.)

Also, the catalog should work with both ESD and WIM files. If the image contains only a single edition, OSD defaults to using Index=1, so there's no need to specify the index manually in that case.

As you mentioned, this PR is specifically for OSDCloud v1, and it's designed to work with Start-OSDCloudGUI.

Thanks for pointing out the new GUI in OSD.Workspace invoked via Start-OSDCloudWorkflow — this is new to me, and I’ll definitely look into it further.

Appreciate your feedback as always!

[KatsuhiroWatanabe]

Update: Integrated Enhancements from feature/InintStartNet2 (PR:#288)

This PR now includes additional enhancements originally proposed in feature/InintStartNet2 (PR:#288), cherry-picked for integration:

• Post-Network Script Execution: Enables execution of a custom script after network initialization.
• Optional Suppression of Module Auto-Update: Prevents automatic updates of OSD modules when explicitly disabled via configuration.

These additions complement the offline catalog support introduced in this PR, enabling more deterministic and customizable OSDCloud initialization workflows—especially in offline or controlled environments.

By integrating both, we provide a more complete solution for environments requiring strict control over OSD behavior.

Note: feature/InintStartNet2 remains unchanged and will be reviewed separately. This integration is intended to accelerate adoption of both features in scenarios where they are used together.

[KatsuhiroWatanabe]

Hi @OSDeploy,

Following up on this PR (originally submitted as #278, which was later replaced) and also #288, which builds on this one. I understand things can get busy, so I just wanted to check in and see if there's anything I can do to assist with the review process.

I've been continuously rebasing this branch onto the latest master to keep it up to date and mergeable, using --force-with-lease to maintain a clean and consistent history. Please let me know if any further changes are needed.

Just to clarify — this PR does not alter existing behavior unless the new functionality is explicitly enabled through additional configuration. It should be safe to merge without impacting current users, and I hope it can be considered for inclusion.

Appreciate all your work on the project!

[KatsuhiroWatanabe]

Hi @OSDeploy,

I've been continuously rebasing this branch onto the latest master to keep it up to date and mergeable. Please let me know if any further changes are needed.

[KatsuhiroWatanabe]

Hi @OSDeploy,

I've been continuously rebasing this branch onto the latest master to keep it up to date and mergeable. Please let me know if any further changes are needed.

[KatsuhiroWatanabe]

Hi @OSDeploy,

I've been continuously rebasing this branch onto the latest master to keep it up to date and mergeable. Please let me know if any further changes are needed.