Skip to content
Merged

Develop #1300

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
216 commits
Select commit Hold shift + click to select a range
16ec967
Initial port of LUD w/ spectral norm constraint.
j-c-c Dec 10, 2024
f1fb779
Use pop in init
j-c-c Dec 10, 2024
28c0d31
Add initial test file.
j-c-c Dec 11, 2024
8118936
remove unused import
j-c-c Dec 11, 2024
246700b
build sparse matrix directly for ComputeATy.
j-c-c Dec 12, 2024
08ae325
init docstring
j-c-c Dec 18, 2024
32fe7a6
reformat
j-c-c Dec 18, 2024
2f44d94
docstrings
j-c-c Jan 6, 2025
9457f13
code comments
j-c-c Jan 7, 2025
2be3439
Move deterministic rounding method to base class
j-c-c Jan 7, 2025
99fff8a
Method for spectral norm constraint subproblem, _compute_Z.
j-c-c Jan 9, 2025
7a911d5
Logic for ADMM w/o spectral norm constraint. Still needs validating a…
j-c-c Jan 9, 2025
9dbcebb
snake case
j-c-c Jan 10, 2025
ebe6d73
cleanup
j-c-c Jan 13, 2025
099336a
tox
j-c-c Jan 14, 2025
dcb771c
staticmethod and docs for compute_AX/ATy
j-c-c Jan 14, 2025
c530904
dtype pass through
j-c-c Jan 14, 2025
2862e88
try upcast for eigs
j-c-c Jan 15, 2025
4a0603c
one more upcast
j-c-c Jan 15, 2025
e4c977a
init parameters
j-c-c Jan 16, 2025
1d8098e
Add 50S ribosome to downloader
j-c-c Jan 22, 2025
1e23c38
add logger message for ADMM iterations. Change ADMM tol for tests (cu…
j-c-c Feb 10, 2025
cabd513
vectorize _Q_theta
j-c-c Feb 25, 2025
ea9ff04
small cleanup
j-c-c Feb 25, 2025
45244a4
add gallery experiment
j-c-c Feb 25, 2025
19243b6
blank line
j-c-c Feb 25, 2025
a713bf7
remove aligned_mse.
j-c-c Feb 27, 2025
2ed1f0a
remove import
j-c-c Feb 27, 2025
7f9451f
add docstring params
j-c-c Feb 27, 2025
50c19c2
src caching for test. remove gpu skip
j-c-c Mar 6, 2025
78c4760
unused import
j-c-c Mar 6, 2025
4e16d87
use default max_shift/shift_step. run doubles always.
j-c-c Mar 6, 2025
def2853
separate fixture for orient_est
j-c-c Mar 6, 2025
3f4a31b
add loop and results table to gallery
j-c-c Mar 6, 2025
298aab5
remove emdb_51751
j-c-c Mar 7, 2025
982f0aa
doubles expensive
j-c-c Mar 7, 2025
41bad44
remove unused method lud_prep
j-c-c Mar 7, 2025
7f3622d
remove spectral_norm_constraint attribute. Just use alpha
j-c-c Mar 7, 2025
52d30a0
cleanup logger message logic
j-c-c Mar 7, 2025
d47365b
tox
j-c-c Mar 7, 2025
deb5e36
Test branches missed in codecov
j-c-c Mar 10, 2025
b7d1a5b
eigenvalue mask
j-c-c Mar 10, 2025
579dd8f
use eigsh (instead of eigs) on symmetric matrices
j-c-c Mar 11, 2025
08fe0dd
compute_num_eigs method
j-c-c Mar 12, 2025
d98f592
Clarifying comments and unit test for compute_AX
j-c-c Mar 13, 2025
dc33d61
Cleanup compute_AX. add docstring to test.
j-c-c Mar 13, 2025
b51f178
refactor compute_num_eigs. Test compute_num_eigs.
j-c-c Mar 13, 2025
f05c84c
num images in gallery: 500
j-c-c Mar 14, 2025
8313bf4
simppler log message
j-c-c Mar 14, 2025
f44e7da
move deterministic_rounding back to SDP. make standalone
j-c-c Mar 14, 2025
8b7d826
collapse logic branches
j-c-c Mar 14, 2025
9c09647
clean up gallery result dict and table
j-c-c Mar 14, 2025
00b7d81
clean up ATy computation
j-c-c Mar 14, 2025
ea5f458
Add ATy to unit test.
j-c-c Mar 14, 2025
bcf8a1d
Fix warning
j-c-c Mar 14, 2025
018f186
use Fraction in gallery
j-c-c Mar 14, 2025
ff0d1a1
unused import
j-c-c Mar 14, 2025
5ddd34a
Fix test tol. Cleanup branch logic.
j-c-c Mar 17, 2025
fd7b6e4
gallery typos
j-c-c Mar 26, 2025
aee6441
commonline_lud review cleanup
j-c-c Mar 27, 2025
db22ef8
test adjoint property
j-c-c Mar 27, 2025
c628511
numpy allclose defaults
j-c-c Mar 27, 2025
79a4618
remove global declaration for read only use of module level variable
j-c-c Apr 1, 2025
4b85dd9
bump ubuntu version in CI
j-c-c Apr 22, 2025
90c0bb4
private Image._load for raw data. Allow rectangular micrographs. Test…
j-c-c Apr 1, 2025
6d9a73b
Switch from raise to log warning for voxel size mismatch
j-c-c Apr 1, 2025
57d085f
remove stack axis for singleton Image.save
j-c-c Apr 1, 2025
d4ae0c0
Cleanup warning in caplog
j-c-c Apr 2, 2025
6cf1c24
remove floating points from assert message
j-c-c Apr 2, 2025
ce05137
remove f-string
j-c-c Apr 2, 2025
9613a2b
_load_raw
j-c-c Apr 22, 2025
1ee71fe
legacy flag for image/volume downsample
j-c-c Mar 28, 2025
c6a75c4
test project/dowsample property with legacy downsample.
j-c-c Mar 28, 2025
105aa8c
test legacy downsample
j-c-c Mar 28, 2025
1d30675
legacy flag smoke test. docstring cleanup.
j-c-c Apr 2, 2025
df82ff3
add full set of parameters to ImageSource.downsample
j-c-c Apr 4, 2025
5e7413a
fix testing logic branch.
j-c-c Apr 8, 2025
9fd54fd
update pyproject license syntax
j-c-c Apr 10, 2025
dbad10e
GPL-3.0-only
j-c-c Apr 23, 2025
28b55e0
Add seed param to BOT_align
j-c-c Apr 29, 2025
6ad3ad0
seed initial point
j-c-c Apr 29, 2025
a95be86
normalize_background legacy mode
j-c-c Apr 7, 2025
1a02baa
custom grid for legacy normalize_bg
j-c-c Apr 9, 2025
81b941d
Add custom grid to test
j-c-c Apr 9, 2025
ab9998a
remove normalization adjustment for even/shifted/normalized grids. Ad…
j-c-c Apr 25, 2025
d3cfbfa
Remove custom grids from Image.normalize_bg and test.
j-c-c Apr 25, 2025
672a4e1
disable -> disabled
j-c-c May 6, 2025
1437a37
Add a test reproducing the oriented source save_mode bug.
garrettwrong May 6, 2025
72aed20
fix save_mode bug
garrettwrong May 6, 2025
fee9bf4
Remove Numpy2 warning
garrettwrong May 8, 2025
0c1dab9
patch volume rotate dtype, issue #1274
garrettwrong May 15, 2025
8eb1a93
added I symmetry and tests for symmetry_groups
May 9, 2025
90da852
added ISymmetryGroup to volume __init__.py
May 11, 2025
a59d769
Get pixel_size from STAR file for RelionSource
j-c-c May 6, 2025
5446351
use file with pixel size metadata for user provided pixel size
j-c-c May 7, 2025
d016d2e
remove unused caplog
j-c-c May 7, 2025
845a750
add pixel size warning
j-c-c May 9, 2025
3da7c36
proceed with download on hash mismatch, with warning.
j-c-c May 15, 2025
51e9d7f
update hash in registry
j-c-c May 15, 2025
3bbe7c8
stack_level. unused variable
j-c-c May 16, 2025
888fd43
add scheduled_workflow and scheduled downloader test.
j-c-c May 16, 2025
3da38ed
same py version
j-c-c May 16, 2025
fd8aa2c
Use logger warning. Update test to fail on hash mismatch warning.
j-c-c May 20, 2025
c1795ae
Workflow updates: Run on develop, remove check, remove fail on warnings.
j-c-c May 20, 2025
7674865
Workflow updates: Checkout develop, set cron off-hour, remove needs f…
j-c-c May 20, 2025
bae7451
test hash mismatch warning works
j-c-c May 20, 2025
ea19605
checkout on develop. mark test as scheduled.
j-c-c May 21, 2025
88b2110
remove manual run-workflow button
j-c-c May 21, 2025
486bfbd
update windows runner
j-c-c Jun 3, 2025
865d3a1
fix dtype
j-c-c Jun 9, 2025
3f8dbea
one more
j-c-c Jun 9, 2025
8e7d26d
cast as python float
j-c-c Jun 10, 2025
c255119
oof
j-c-c Jun 10, 2025
3be1b85
Cn/Dn create_group. Adapt D2 use of DnSymmetryGroup.
j-c-c May 27, 2025
23cb770
T/O create group
j-c-c May 27, 2025
3c495e9
use to_string
j-c-c May 27, 2025
87d2274
Remove redundant code
j-c-c May 29, 2025
5c14652
add ISymmetricVolume
j-c-c May 29, 2025
cc68527
typo
j-c-c May 29, 2025
9fa3ccc
g_sync symm group order
j-c-c Jun 4, 2025
5bfa1b9
remove uniform_random_angles
j-c-c May 22, 2025
9c771b1
refactor mean_aligned_angular_distance
j-c-c May 23, 2025
937610b
remove duplicate rotation registration code
j-c-c May 23, 2025
b994c6b
only convert to Rotation if necessary
j-c-c May 27, 2025
9bfd388
Remove duplicate common_line_from_rots code
j-c-c May 27, 2025
254b7ed
update docstring
j-c-c Jun 3, 2025
8af3314
permit corrupt MRC files
garrettwrong Jun 11, 2025
9ab9d96
stub in BFT work from notebook
garrettwrong Mar 27, 2025
b6a9342
fixup mixing with translations
garrettwrong Mar 27, 2025
4083c0e
vector fast polar align
garrettwrong Mar 28, 2025
7617386
shift base image and commute shift
garrettwrong Mar 28, 2025
4043179
cleanup
garrettwrong Mar 28, 2025
54be457
hack in gpu code, dirty
garrettwrong Mar 28, 2025
d5af43a
factor out the pft
garrettwrong Mar 28, 2025
32d56bf
begin batching, two places to broadcast
garrettwrong Mar 28, 2025
e9d44bd
table broadcast polar cross corr
garrettwrong Mar 28, 2025
b86ee43
table broadcast shifts, resuse arrays, reduce mem cost some speed
garrettwrong Mar 28, 2025
15e6a9b
Cleanup unit test for broadcast case
garrettwrong Apr 23, 2025
491e951
cleanup pft interop
garrettwrong Apr 23, 2025
1e471bc
A little more cleanup
garrettwrong Apr 23, 2025
eb89388
stash
garrettwrong Apr 23, 2025
a6f8af1
add fine interp and optimize methods
garrettwrong Apr 24, 2025
2d7d8cb
add BFTAverager2D to test suite
garrettwrong Apr 25, 2025
ee5f5cb
intial add BFT to source wrappers, remove 110
garrettwrong Apr 28, 2025
4ecbc53
tox checks
garrettwrong Apr 29, 2025
b37f3c7
flip bug fix
garrettwrong Apr 29, 2025
b3cd1cf
update shift grid to return array of tuples
garrettwrong Apr 30, 2025
b3b077d
cleanup
garrettwrong May 1, 2025
8d9e744
reversed the index mapping, whoops
garrettwrong May 1, 2025
67dcf40
copy syntax
garrettwrong May 1, 2025
1ce1b1a
remove interp option from polar cross cor align
garrettwrong May 8, 2025
889543f
cleanup comment
garrettwrong May 13, 2025
d3d111f
update //16 to //32 in shift search
garrettwrong May 15, 2025
24c5c8b
default to self.n_radial
garrettwrong May 15, 2025
3bd578b
typo ceates -> creates
garrettwrong May 15, 2025
08a19cc
docstring updates
garrettwrong May 15, 2025
9c5a9dc
use L//2 for n_radial
garrettwrong May 16, 2025
4c6fedd
len(shifts) ~> len(test_shifts)
garrettwrong May 21, 2025
a1f2043
cleanup minor review remarks
garrettwrong May 21, 2025
ac0c073
sub pixel review change bug
garrettwrong May 23, 2025
f9ed410
stub in PolarFT shifting
garrettwrong May 27, 2025
cbb6bf3
stub in PolarFT shift test 2d
garrettwrong May 28, 2025
9a4b52a
add broadcast polar shift test
garrettwrong May 28, 2025
f147b25
add multiple shift broadcast polar code
garrettwrong May 28, 2025
87a9f48
Use PolarFT.shift in BFT class source
garrettwrong May 28, 2025
8ad361b
exted PolarFT.shift to xp
garrettwrong May 28, 2025
83ac369
update pinned cu/finufft to 2.4.0
garrettwrong Jun 25, 2025
e04c0ea
let's try updating the conda envs along with finufft
garrettwrong Jun 26, 2025
4c39b51
revert finufft for darwin
garrettwrong Jun 30, 2025
b16ae0d
revert finufft upsampfac
garrettwrong Jun 30, 2025
ceb6cbd
Revert env yamls
garrettwrong Jul 1, 2025
3878314
add more pixel_size attribute handling to CoordinateSources
garrettwrong Jun 24, 2025
ba325f4
update utest to approx_equal
garrettwrong Jun 25, 2025
e61c015
cleanup strings
garrettwrong Jun 25, 2025
567a38b
add multiple pixel size test
garrettwrong Jun 25, 2025
6eea91a
fix strings spelling typos
garrettwrong Jun 26, 2025
6b45738
replace list(set(...)) with np.unqiue(...)
garrettwrong Jun 26, 2025
04c9c04
Update comment
garrettwrong Jul 1, 2025
d41cce9
Initial port of IRLS w/ spectral norm constraint.
j-c-c Apr 28, 2025
15424d5
Add basic test
j-c-c Apr 28, 2025
7726617
tox
j-c-c Apr 28, 2025
cbef6c0
use 5 degree tol
j-c-c Apr 29, 2025
732fe61
fix adp_proj branch
j-c-c May 2, 2025
68e10f3
Add branch for no-spectral-norm-constraint. Add testing.
j-c-c May 2, 2025
f678f8d
remove ctype (dead branch in matlab).
j-c-c May 7, 2025
8744b14
Adapt lud compute_Z and use for IRLS
j-c-c May 8, 2025
9dee720
Adjoint property test
j-c-c May 8, 2025
8a1172b
add IRLS to LUD gallery
j-c-c May 12, 2025
40c4cbc
edit gallery
j-c-c May 16, 2025
56bff84
docstrings
j-c-c May 16, 2025
c8e3546
Rearrange class structure
j-c-c May 21, 2025
64609da
add comment about self._mu
j-c-c Jun 12, 2025
635e402
update class docstring
j-c-c Jun 12, 2025
76d8949
refer parent class for argument docstring
j-c-c Jun 12, 2025
6916d6d
S -> S_weighted
j-c-c Jun 13, 2025
493ef77
use broadcasting
j-c-c Jun 13, 2025
b715702
eigenvalue eps for singles/doubles
j-c-c Jun 13, 2025
9ad9f03
variable name
j-c-c Jun 13, 2025
14096cd
update docstring
j-c-c Jun 16, 2025
a914fd1
Fix compute_AX/compute_ATy docstrings to distinguish between LUD/IRLS.
j-c-c Jun 16, 2025
bd35533
add a pbar
j-c-c Jun 16, 2025
8c7b845
use *args
j-c-c Jun 16, 2025
cd93f26
_compute_gram_matrix ->_compute_gram_SDP
j-c-c Jul 1, 2025
52d3ce6
_compute_Gram ->_compute_gram_IRLS
j-c-c Jul 1, 2025
1685878
resolve ambiguous dtype in Downsample
garrettwrong Jul 3, 2025
5cd8bc7
Update comment with GH issue
garrettwrong Jul 3, 2025
0afa283
Bump version: 0.13.2 → 0.14.0
garrettwrong Jul 3, 2025
33e24af
update pipeline demo
j-c-c Jun 26, 2025
092cd3e
tox
j-c-c Jun 26, 2025
de35baa
typo/grammar
j-c-c Jun 27, 2025
8d56595
Remove BOT. Align with Q_mat
j-c-c Jun 30, 2025
d4a42c6
Add additional preprocessing
j-c-c Jul 1, 2025
9650b90
Comment cleanup
j-c-c Jul 1, 2025
f067b3c
Cache at every stage. Fix alignment flip/rotate order
j-c-c Jul 1, 2025
990d666
cache sim, post preprocessing, post averages
j-c-c Jul 2, 2025
60d1db6
fix sphinx cell ignore. estimate rots in Orientation Estimation block.
j-c-c Jul 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.13.2
current_version = 0.14.0
commit = True
tag = True

Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/scheduled_workflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: ASPIRE Python Scheduled Workflow

on:
schedule:
- cron: '15 0 * * 0' # Every Sunday at 00:15 UTC


jobs:
scheduled-tests:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
ref: develop
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install Dependencies
run: |
pip install -e ".[dev]"
- name: Scheduled Tests
run: pytest -m scheduled
4 changes: 2 additions & 2 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,15 @@ jobs:
shell: bash -el {0}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-20.04, macOS-latest, macOS-13]
os: [ubuntu-latest, ubuntu-22.04, macOS-latest, macOS-13]
backend: [default, openblas]
python-version: ['3.9']
include:
- os: ubuntu-latest
backend: intel
- os: macOS-latest
backend: accelerate
- os: windows-2019
- os: windows-2022
backend: win64

steps:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5657281.svg)](https://doi.org/10.5281/zenodo.5657281)
[![Downloads](https://static.pepy.tech/badge/aspire/month)](https://pepy.tech/project/aspire)

# ASPIRE - Algorithms for Single Particle Reconstruction - v0.13.2
# ASPIRE - Algorithms for Single Particle Reconstruction - v0.14.0

The ASPIRE-Python project supersedes [Matlab ASPIRE](https://github.com/PrincetonUniversity/aspire).

Expand All @@ -20,7 +20,7 @@ For more information about the project, algorithms, and related publications ple
Please cite using the following DOI. This DOI represents all versions, and will always resolve to the latest one.

```
ComputationalCryoEM/ASPIRE-Python: v0.13.2 https://doi.org/10.5281/zenodo.5657281
ComputationalCryoEM/ASPIRE-Python: v0.14.0 https://doi.org/10.5281/zenodo.5657281

```

Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@
# built documents.
#
# The full version, including alpha/beta/rc tags.
release = version = "0.13.2"
release = version = "0.14.0"

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Aspire v0.13.2
Aspire v0.14.0
==============

Algorithms for Single Particle Reconstruction
Expand Down
113 changes: 113 additions & 0 deletions gallery/experiments/commonline_lud_simulated_data.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
"""
Commonlines Method using LUD
============================

This tutorial demonstrates using the Least Unsquared Deviations (LUD)
commonlines method for estimating particle orientations. This tutorial
reproduces the "Experiments on simulated images" found in the publication:

Orientation Determination of Cryo-EM Images Using Least Unsquared Deviations,
L. Wang, A. Singer, and Z. Wen, SIAM J. Imaging Sciences, 6, 2450-2483 (2013).
"""

# %%
# Imports
# -------

import logging
from fractions import Fraction
from itertools import product

import numpy as np

from aspire.abinitio import CommonlineIRLS, CommonlineLUD
from aspire.noise import WhiteNoiseAdder
from aspire.source import Simulation
from aspire.utils import mean_aligned_angular_distance
from aspire.volume import Volume

logger = logging.getLogger(__name__)


# %%
# Parameters
# ----------
# Set up some initializing parameters. We will run the LUD algorithm using ADMM
# and IRLS methods under various spectral norm constraints and levels of noise.

SNR = ["1/8", "1/16", "1/32"] # Signal-to-noise ratio
METHOD = ["ADMM", "IRLS"]
ALPHA = [0.90, 0.75, 0.67] # Spectral norm constraint
n_imgs = 500 # Number of images in our source
dtype = np.float64
pad_size = 129

# %%
# Load Volume Map
# ---------------
# We will generate simulated noisy images from a low res volume
# map available in our data folder. This volume map is a 65 x 65 x 65
# voxel volume which we intend to upsample to 129 x 129 x 129.
# To do this we use our ``downsample`` method which, when provided a voxel
# size larger than the input volume, internally zero-pads in Fourier
# space to increase the overall shape of the volume.
vol = (
Volume.load("../tutorials/data/clean70SRibosome_vol_65p.mrc")
.astype(dtype)
.downsample(pad_size)
)
logger.info("Volume map data" f" shape: {vol.shape} dtype:{vol.dtype}")

# %%
# Generate Noisy Images and Estimate Rotations
# --------------------------------------------
# A ``Simulation`` object is used to generate simulated data at various
# noise levels. Then rotations are estimated using the ``CommonlineLUD`` and
# ``CommonlineIRLS`` algorithms. Results are measured by computing the mean
# aligned angular distance between the ground truth rotations and the globally
# aligned estimated rotations.

# Build table to dislay results.
col_width = 21
table = []
table.append(
f"{'METHOD':<{col_width}} {'SNR':<{col_width}} {'ALPHA':<{col_width}} {'Mean Angular Distance':<{col_width}}"
)
table.append("-" * (col_width * 4))

for method, snr, alpha in product(METHOD, SNR, ALPHA):
# Generate a white noise adder with specified SNR.
noise_adder = WhiteNoiseAdder.from_snr(snr=Fraction(snr))

# Initialize a Simulation source to generate noisy, centered images.
src = Simulation(
n=n_imgs,
vols=vol,
offsets=0,
amplitudes=1,
noise_adder=noise_adder,
dtype=dtype,
).cache()

# Estimate rotations using the LUD algorithm.
if method == "ADMM":
orient_est = CommonlineLUD(src, alpha=alpha)
else:
orient_est = CommonlineIRLS(src, alpha=alpha)
est_rotations = orient_est.estimate_rotations()

# Find the mean aligned angular distance between estimates and ground truth rotations.
mean_ang_dist = mean_aligned_angular_distance(est_rotations, src.rotations)

# Append results to table.
table.append(
f"{method:<{col_width}} {snr:<{col_width}} {str(alpha):<{col_width}} {mean_ang_dist:<{col_width}}"
)

# %%
# Display Results
# ---------------
# Display table of results for both methods using various spectral norm
# constraints and noise levels.

logger.info("\n" + "\n".join(table))
Loading
Loading