Skip to content

Conversation

@dblythy
Copy link
Member

@dblythy dblythy commented Jan 19, 2023

New Pull Request Checklist

Issue Description

It's currently only possible to export 10,000 rows, and only as a CSV

Closes: #1961

Approach

Uses Parse.Query.eachBatch to paginate over full class and exports as files when:

  • Array of objects is over 1gb in memory
  • New objects length is less than batch size (eachBatch has finished)

Screenshot 2023-01-19 at 3 45 34 pm

TODOs before merging

  • Add tests
  • Add changes to documentation (guides, repository pages, in-code descriptions)

@parse-github-assistant
Copy link

parse-github-assistant bot commented Jan 19, 2023

Thanks for opening this pull request!

  • 🎉 We are excited about your hands-on contribution!

@dblythy dblythy requested a review from a team January 19, 2023 05:40
mtrezza
mtrezza previously approved these changes Jan 20, 2023
Copy link
Member

@mtrezza mtrezza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@dblythy
Copy link
Member Author

dblythy commented Jan 22, 2023

Screen Shot 2023-01-23 at 10 19 15 am

@dblythy dblythy requested a review from a team January 22, 2023 23:29
Copy link
Member

@mtrezza mtrezza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice rework of the dialog!

I looked at the delete rows dialog, to use the same wording here:

Confirm this action
Enter "export all" to continue.

Unlike a delete action, it may not be clear why this is a critical operation. So in addition we could add a text that says:

⚠️ Exporting all rows may significantly impact resources.

Maybe you could add that below the "Estimated data size", or if it looks acceptable add it to the confirmation segment:

Confirm this action
⚠️ Exporting all rows may significantly impact resources. Enter "export all" to continue.

@dblythy
Copy link
Member Author

dblythy commented Jan 24, 2023

Screenshot 2023-01-25 at 10 51 31 am

How's this look?

Copy link
Member

@mtrezza mtrezza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mtrezza mtrezza changed the title feat: Allow export all columns, allow export as JSON feat: Add export all rows of a class and export in JSON format Jan 25, 2023
@mtrezza mtrezza merged commit 9eb36a1 into parse-community:alpha Jan 25, 2023
@dblythy dblythy deleted the download-all branch January 25, 2023 13:22
@dblythy
Copy link
Member Author

dblythy commented Jan 25, 2023

Thanks for your help!

parseplatformorg pushed a commit that referenced this pull request Jan 25, 2023
# [5.1.0-alpha.5](5.1.0-alpha.4...5.1.0-alpha.5) (2023-01-25)

### Features

* Add export all rows of a class and export in JSON format ([#2361](#2361)) ([9eb36a1](9eb36a1))
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.1.0-alpha.5

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Jan 25, 2023
parseplatformorg pushed a commit that referenced this pull request Mar 1, 2023
# [5.1.0-beta.2](5.1.0-beta.1...5.1.0-beta.2) (2023-03-01)

### Bug Fixes

* Add dashboard option `cookieSessionMaxAge` to keep user logged in across browser sessions ([#2366](#2366)) ([9ea95fc](9ea95fc))
* Blank screen shown if server is unreachable; unsupported pages are accessible via direct URLs ([#2363](#2363)) ([9855258](9855258))
* Dashboard may display blank page when selecting an app after login ([#2375](#2375)) ([f399b91](f399b91))
* Data browser dialog "No data to display" may be outside of visible area in Safari browser ([#2387](#2387)) ([52bba62](52bba62))
* Internal error message on login with missing credential ([#2370](#2370)) ([9a6a31f](9a6a31f))
* Navigation to page fails if user re-login is required ([#2369](#2369)) ([0db6f55](0db6f55))
* Screen goes blank when trying to add column of type `Object` or `GeoPoint` ([#2384](#2384)) ([0886386](0886386))
* Text selection not visible in modal dialog header ([#2340](#2340)) ([fb0e79c](fb0e79c))

### Features

* Add export all rows of a class and export in JSON format ([#2361](#2361)) ([9eb36a1](9eb36a1))
* Add schema export ([#2362](#2362)) ([33df049](33df049))
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.1.0-beta.2

@parseplatformorg parseplatformorg added the state:released-beta Released as beta version label Mar 1, 2023
parseplatformorg pushed a commit that referenced this pull request May 1, 2023
# [5.1.0](5.0.0...5.1.0) (2023-05-01)

### Bug Fixes

* Add dashboard option `cookieSessionMaxAge` to keep user logged in across browser sessions ([#2366](#2366)) ([9ea95fc](9ea95fc))
* Blank screen shown if server is unreachable; unsupported pages are accessible via direct URLs ([#2363](#2363)) ([9855258](9855258))
* Dashboard may display blank page when selecting an app after login ([#2375](#2375)) ([f399b91](f399b91))
* Data browser dialog "No data to display" may be outside of visible area in Safari browser ([#2387](#2387)) ([52bba62](52bba62))
* Internal error message on login with missing credential ([#2370](#2370)) ([9a6a31f](9a6a31f))
* Navigation to page fails if user re-login is required ([#2369](#2369)) ([0db6f55](0db6f55))
* Screen goes blank when trying to add column of type `Object` or `GeoPoint` ([#2384](#2384)) ([0886386](0886386))
* Text selection not visible in modal dialog header ([#2340](#2340)) ([fb0e79c](fb0e79c))

### Features

* Add export all rows of a class and export in JSON format ([#2361](#2361)) ([9eb36a1](9eb36a1))
* Add schema export ([#2362](#2362)) ([33df049](33df049))
* remove limitation to refresh Cloud Jobs list only after 30 seconds ([#2332](#2332)) ([ad1132f](ad1132f))
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.1.0

@parseplatformorg parseplatformorg added the state:released Released as stable version label May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can't export more than 10k rows as csv

3 participants