🚀 Switchover from Trac to GitHub

[mkoeppe]

0. Announce switchover schedule on sage-devel.
1. Jan 30, 13:00 UTC: Make Trac read-only (this needs help by a Trac admin):
   • Take Trac offline,
   • Reconfigure Trac ticket system, Trac wiki to be read-only
   • Make sure Trac git repo is fully synced to https://github.com/sagemath/sagetrac-mirror.
   • Make Trac git repo read-only
   • Bring Trac online.
   • Announce read-only availability of Trac on sage-devel
   • this needs to be completed by Jan 30, 23:00 UTC the latest
2. Archive sagemath/sagetrac-mirror (= set it to readonly).
3. Final update of GitHub usernames.
4. Re-run write-closing-commits.sh (done in 83bc2e2, includes 9.8.rc0)
5. Use fresh worktree worktree-prod (instead of clearing the trac-to-github disk cache (rm -rf trac_cache) and the migration archive (git clean -fX archive))
6. Run migrate.py to create the migration archive in archive/. about 6.5 hours runtime, needs running, read-only Trac server
7. Set migrate.cfg.sagetracmigrationarchive [wiki] migrate: yes
8. Check any remaining pull requests for the migration script, merge master into production. Clear the migration archive again (git clean -fX archive).
9. Re-run migrate.py. about 30 minutes.
10. (Optional:) put a mirrored clone of sagemath/sage into the migration archive (but remove all branches except master, develop!) at archive/repositories/sagemath/sage.git. In the bare repo, run rm -Rf refs/pull && git gc to get rid of the pull request refs from sagemath/sage; we do not transfer these PRs in this step. (Alternatively, we can push to the repo later.)
11. Init the bare repository archive/repositories/sagemath/sage.wiki.git. (Importing a populated wiki repository does not seem to work – wiki at https://34.105.185.241/sagemath/sage-20230130181123/wiki was empty on import. So we push the wiki later.)
12. Edit the repo description of sagemath/sage as preparation for renaming/archiving it.
13. (Optional:) Import into our GHE server for checking for necessary additions to map-ghe.csv.in and general inspection. about 20 hours runtime.
14. Rename sagemath/sage to sagemath/sage-archive-2023-02-01.
15. Archive sagemath/sage-archive-2023-02-01.
16. Feb 1, 9am Pacific Time: Request access to Enterprise Cloud Importer. (this needs help by our GitHub contacts)
17. Upload the migration archive archive.tar.gz; also upload map-github.csv (edited from map-ghe.csv.in -- see script rsync-archive-to-ghe-and-import.sh).
18. Set org Member permissions to minimal. Invite users to sagemath GitHub org using API (get-a-user, create-org-invite), post heads-up on sage-devel, create org README as a landing page for invitees
19. Feb 3, 0:00 UTC: Fix up the map using GraphQL (ECI does not allow me to go back to change the map when it thinks it is "READY") or restart the import from scratch (~1h).
20. Import ("perform") – as a new repo sagemath/sage (this should break the redirect sagemath/sage -> sagemath/sage-archive-2023-02-01).
    4. (Optional:) Create a single-branch fork of sagemath/sage-temp called sagemath/sagetrac-mirror and push all branches (or all branches of open tickets) from sagemath/sagetrac-archive to it. Open PRs from sagemath/sagetrac-mirror to sagemath/sage-temp for all open tickets with attached branches.
21. Archive the repo so that we don't catch premature PRs
22. Announce preview on sage-devel.
23. Rename sagemath/sage to sagemath/sage-migration-preview-2023-02-02
24. Fix remaining attachment issues (Fix up handling of repository_files / attachments for some types #176)
25. Feb 4, 22:00 UTC Regenerate archive again, import as sagemath/sage-prod
26. Rename faulty import sagemath/sage to sagemath/sage-migration-preview-2023-02-04
27. rename sagemath/sage-prod to sagemath/sage
28. Disable Actions until Disable pull-request github workflows #157 is done (6526184)
    17. [ ] Rename the new repo sagemath/sage-temp to sagemath/sage. (Yes, this will work and will break the redirect sagemath/sage -> sagemath/sage-archive-2023-02-01.)
    18. [ ] (Optional:) Transfer the existing (few) issues from sagemath/sage-archive-2023-02-01 to sagemath/sage.
29. fast-forward develop to 9.8.rc1,
30. enable Actions,
31. check that Projects are disabled
32. disable Discussions
33. planned: by Feb 6, 08:00 UTC Announce on sage-devel that sagemath/sage is now open for Issues and PRs.
34. Clone https://github.com/sagemath/sage-archive-2023-02-01.wiki, add converted wiki. merged-wiki
35. Push to https://github.com/sagemath/sage.wiki
36. Display a link from each Trac ticket to the converted GitHub Issue (this needs help by a Trac admin)

[tobiasdiez]

I'm a bit out of the loop about how exactly open tickets with existing branches are now converted (issue or pr?), but in either way the migration guide should document how to keep working on an existing trac ticket/branch.

[mkoeppe]

Per discussion in #57, trying to have the migration script create PRs is probably not worth the effort.

#68 asks to document the procedure to start a PR from a migrated issue with an existing branch.

[kwankyu]

Where is the procedure to transfer trac wiki pages?

Like the practice of using the script, perhaps we should migrate the wiki first and then tickets, for safety.

[mkoeppe]

The procedure depends on the decision made in #83

[kwankyu]

Convert all tickets to Issues in a new repo sagemath/sage-temp via a migration archive.

Do we use the migration archive (the json files in /archive) for this? As I understand it, we use the migration archive only for GitHub Enterprise Server.

[mkoeppe]

Yes, we will use the migration archive for the final import too.

[kwankyu]

Sorry about asking perhaps a settled matter.

According to the documentation like https://docs.github.com/en/enterprise-server@3.7/admin/user-management/migrating-data-to-and-from-your-enterprise, I understand that migration archive is for migration to GitHub Enterprise Server, but not to github.com. Does github.com provide an API for migration archive? Did we test it? Or do we have a special agreement with github.com?

[mkoeppe]

We have a contact at GitHub who will take care of the import of the migration archive for us.

[kwankyu]

OK! Thanks.

[kwankyu]

If we are on the schedule as planned here, an announcement of the migration day and the schedule should be made on sage-devel. Perhaps a notice of what an individual developer should do, if any, may be included.

[mkoeppe]

Yes, I'm waiting for a Trac admin to confirm that they are able to do step 1 by Jan 30. @dimpase

[dimpase]

yes, sure, I'll do this in some way. Let's say I'll start at 13:00 GMT on 30 Jan, and hope to be done by 23:00 GMT (maybe earlier).

I wish we had @embray helping though - he's an expert in trac...

[mkoeppe]

Great, thank you!

I'll send out an announcement on sage-devel.