Skip to content

Conversation

acwhite211
Copy link
Member

@acwhite211 acwhite211 commented Sep 25, 2025

Fixes #3970

Create a patch migration to de-duplicate and merge picklists. Before deleting duplicates, merge the picklist items into one of the picklists to make sure no items are missing. This is to resolve duplicate picklists that were created in Specify 6.

For the purposes of this PR, we have decided that two picklists are considered duplicates if they are identical on all the following fields:

  • name
  • tablename
  • fieldname
  • collectionid

Unfortunately, picklists are only differentiated by name in the schema and workbench. Also, there might be cases where two picklists have the same name, but have different types/tables/etc. Those would be special cases that would be difficult to categorize as duplicates or not, depending on the user's intent. This PR focuses I identifying, deduplicating, and merging picklists that are certain to be duplicates based on those four fields.

Might want to also add this patch migration to the key migrations command.

When merging together picklistitems from a group of duplicate picklists, the following fields are compared:

  • title
  • value

Checklist

  • Self-review the PR after opening it to make sure the changes look good and
    self-explanatory (or properly documented)
  • Add relevant issue to release milestone
  • Add pr to documentation list

Testing instructions

  • Use a database with known duplicate picklists and start it up on a new instance so that this new patch migration is applied. The picklist that is duplicated by Specify 6 is the 'TypeStatus' picklist, so make sure to check for that one in the testing. naturkundemuseum is a database with known picklist duplicates.
  • Open up the schema config tool and see that the picklist has be deduplicated.
  • For dev testing, duplicates can be found with this query. Ensure that it returns no records after the migration runs.
select pl.Name, pl.TableName, pl.FieldName, c.CollectionName, count(*) as pl_count
from picklist pl
left outer join collection c on c.UserGroupScopeId = pl.CollectionID
group by pl.Name, pl.TableName, pl.FieldName, pl.CollectionID
having count(*) > 1
order by pl_count desc;

@acwhite211
Copy link
Member Author

@grantfitzsimmons Can you see if this PR removes the enough problematic picklist duplicates? Just want to make sure before progressing to further testing.

@acwhite211 acwhite211 marked this pull request as ready for review October 2, 2025 21:40
@acwhite211 acwhite211 self-assigned this Oct 2, 2025
Added import for Count to facilitate deduplication.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 📋Back Log

Development

Successfully merging this pull request may close these issues.

[Guided Setup] - Duplicate pick lists (with the same name) are being created upon database creation

1 participant