Skip to content

Commit 8ffe27f

Browse files
author
R. S. Doiel
committed
prep for 2.2.3.1 release
1 parent 4e942dd commit 8ffe27f

File tree

103 files changed

+3641
-113
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

103 files changed

+3641
-113
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@ py_dataset/lib/README.md
1414
# test collections and files
1515
*.ds
1616
test_*csv
17+
TODO.html

‎CITATION.cff‎

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,38 @@
1+
12
cff-version: 1.2.0
23
message: "If you use this software, please cite it as below."
4+
type: software
35
title: py_dataset
6+
abstract: "A command line tool for working with JSON documents on local disc"
47
authors:
58
- family-names: Doiel
69
given-names: Robert
710
orcid: https://orcid.org/0000-0003-0900-6903
11+
email: [email protected]
812
- family-names: Morrell
913
given-names: Thomas E
1014
orcid: https://orcid.org/0000-0001-9266-5146
11-
abstract: A command line tool for working with JSON documents on local disc
15+
email: [email protected]
16+
17+
contacts:
18+
- family-names: Doiel
19+
given-names: R. S.
20+
orcid: https://orcid.org/0000-0003-0900-6903
21+
email: [email protected]
22+
- family-names: Morrell
23+
given-names: Thomas E
24+
orcid: https://orcid.org/0000-0001-9266-5146
25+
email: [email protected]
26+
1227
repository-code: "https://github.com/caltechlibrary/py_dataset"
13-
type: software
14-
version: 2.1.2
15-
license-url: "https://data.caltech.edu/license"
28+
version: 2.2.3.1
29+
date-released: 2025-04-17
30+
31+
license-url: "https://github.com/caltechlibrary/py_dataset/blob/main/LICENSE"
1632
keywords:
1733
- GitHub
1834
- metadata
1935
- data
2036
- software
2137
- json
22-
date-released: 2023-09-27
38+

‎DEVELOPER-NOTES.html‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
<!DOCTYPE html>
2+
<html lang="en-US">
3+
<head>
4+
<title>py_dataset</title>
5+
<link rel="stylesheet" href="/css/site.css">
6+
</head>
7+
<body>
8+
<nav>
9+
<ul>
10+
<li><a href="/">Home</a></li>
11+
<li><a href="index.html">README</a></li>
12+
<li><a href="LICENSE">LICENSE</a></li>
13+
<li><a href="user_manual.html">User Manual</a></li>
14+
<li><a href="about.html">About</a></li>
15+
<!-- <li><a href="search.html">Search</a></li> -->
16+
<li><a href="https://github.com/caltechlibrary/py_dataset">GitHub</a></li>
17+
</ul>
18+
</nav>
19+
<section>
20+
<h1 id="developer-notes-for-py_dataset">Developer notes for
21+
py_dataset</h1>
22+
<p>py_dataset wraps <a
23+
href="https://github.com/caltechlibrary/dataset">dataset</a>’s
24+
libdataset. We try to keep the version of py_dataset in sync with the
25+
current release of dataset. <del>libdataset. When a new version of
26+
<em>dataset</em> and <em>libdataset</em> are release fetch the
27+
<code>libdataset-*.zip</code> archives and unzip them in
28+
<code>py_dataset/lib</code>. This should leave you with
29+
<code>libdataset.so</code> (Linux), <code>libdataset.dylib</code> (Mac
30+
OS X) and <code>libdataset.dll</code> (Windows) as well as
31+
<code>libdataset.h</code> (same for all three platforms). You will need
32+
to update <code>py_dataset/dataset.py</code> to include any changes you
33+
want to reflect in the shared library (e.g. add new functions). Once
34+
those twothings have been done you can use <code>python setup.py</code>
35+
to install your updates locally and test.</del></p>
36+
<p>For patches between dataset release we append a period and patch
37+
value to the semver that matched dataset’s release. E.g. the first patch
38+
to py_dataset paired with dataset 2.2.3 release would be 2.2.3.1. The
39+
second 2.2.3.2, etc.</p>
40+
<p>py_dataset is now built/managed using <a
41+
href="https://docs.astral.sh/uv/">uv</a>.</p>
42+
<h2 id="release-process">Release process</h2>
43+
<ul>
44+
<li><del>Update shared libraries in <code>py_dataset/lib</code>
45+
(e.g. <code>libdataset.so</code>, <code>libdataset.dynlib</code> and
46+
<code>libdataset.dll</code>)</del></li>
47+
<li>Update <code>py_dataset/dataset.py</code> and
48+
<code>py_dataset/libdataset.py</code> as needed</li>
49+
<li>Update version info in <code>codemeta.json</code>
50+
(e.g. <code>cme version releaseNotes dateModified datePublished -e</code>)</li>
51+
<li>Update README.md and DEVELOPER-NOTES.md</li>
52+
<li>Test and commit changes to the master branch in the git repo
53+
<ul>
54+
<li><code>uv run python3 test_dataset.py</code></li>
55+
</ul></li>
56+
<li>Make a github release if you’re happy with everything</li>
57+
<li>[optional] Push changes up to pypy</li>
58+
</ul>
59+
</section>
60+
</body>
61+
</html>

‎DEVELOPER-NOTES.md‎

Lines changed: 11 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,20 @@
11

22
# Developer notes for py_dataset
33

4-
py_dataset wraps [dataset](https://github.com/caltechlibrary/dataset)'s
5-
libdataset. We try to keep the version of py_dataset in sync with the
6-
current release of libdataset. When a new version of _dataset_ and
7-
_libdataset_ are release fetch the `libdataset-*.zip` archives and
8-
unzip them in `py_dataset/lib`. This should leave you with `libdataset.so`
9-
(Linux), `libdataset.dylib` (Mac OS X) and `libdataset.dll` (Windows)
10-
as well as `libdataset.h` (same for all three platforms). You will need
11-
to update `py_dataset/dataset.py` to include any changes you want to
12-
reflect in the shared library (e.g. add new functions). Once those two
13-
things have been done you can use `python setup.py` to install your
14-
updates locally and test.
4+
py_dataset wraps [dataset](https://github.com/caltechlibrary/dataset)'s libdataset. We try to keep the version of py_dataset in sync with the current release of dataset. ~~libdataset. When a new version of _dataset_ and _libdataset_ are release fetch the `libdataset-*.zip` archives and unzip them in `py_dataset/lib`. This should leave you with `libdataset.so` (Linux), `libdataset.dylib` (Mac OS X) and `libdataset.dll` (Windows) as well as `libdataset.h` (same for all three platforms). You will need to update `py_dataset/dataset.py` to include any changes you want to reflect in the shared library (e.g. add new functions). Once those twothings have been done you can use `python setup.py` to install your updates locally and test.~~
5+
6+
For patches between dataset release we append a period and patch value to the semver that matched dataset's release. E.g. the first patch to py_dataset paired with dataset 2.2.3 release would be 2.2.3.1. The second 2.2.3.2, etc.
7+
8+
py_dataset is now built/managed using [uv](https://docs.astral.sh/uv/).
159

1610
## Release process
1711

18-
+ Update shared libraries in `py_dataset/lib` (e.g. `libdataset.so`, `libdataset.dynlib` and `libdataset.dll`)
19-
+ Update `py_dataset/dataset.py` as needed
20-
+ Update version number in `codemeta.json`
12+
+ ~~Update shared libraries in `py_dataset/lib` (e.g. `libdataset.so`, `libdataset.dynlib` and `libdataset.dll`)~~
13+
+ Update `py_dataset/dataset.py` and `py_dataset/libdataset.py` as needed
14+
+ Update version info in `codemeta.json` (e.g. `cme version releaseNotes dateModified datePublished -e`)
2115
+ Update README.md and DEVELOPER-NOTES.md
2216
+ Test and commit changes to the master branch in the git repo
23-
+ `python3 test_dataset.py`
24-
+ Make a github release
25-
+ Push changes up to pypy
17+
+ `uv run python3 test_dataset.py`
18+
+ Make a github release if you're happy with everything
19+
+ [optional] Push changes up to pypy
2620

‎Makefile‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
#
2+
# Simple Makefile for Golang based Projects built under POSIX.
3+
#
4+
PROJECT = py_dataset
5+
6+
GIT_GROUP = caltechlibrary
7+
8+
PROGRAMS =
9+
10+
RELEASE_DATE = $(shell date +%Y-%m-%d)
11+
12+
RELEASE_HASH=$(shell git log --pretty=format:'%h' -n 1)
13+
14+
MAN_PAGES_1 = #$(shell ls -1 *.1.md | sed -E 's/.1.md/.1/g')
15+
16+
MAN_PAGES_3 = #$(shell ls -1 *.3.md | sed -E 's/.3.md/.3/g')
17+
18+
MAN_PAGES_7 = #$(shell ls -1 *.7.md | sed -E 's/.7.md/.7/g')
19+
20+
HTML_PAGES = $(shell find . -type f | grep -E '.html$')
21+
22+
DOCS = $(shell ls -1 *.?.md)
23+
24+
PACKAGE = $(shell ls -1 *.py)
25+
26+
VERSION = $(shell grep '"version":' codemeta.json | cut -d" -f 4)
27+
28+
BRANCH = $(shell git branch | grep '* ' | cut -d -f 2)
29+
30+
OS = $(shell uname)
31+
32+
#PREFIX = /usr/local/bin
33+
PREFIX = $(HOME)
34+
35+
ifneq ($(prefix),)
36+
PREFIX = $(prefix)
37+
endif
38+
39+
EXT =
40+
ifeq ($(OS), Windows)
41+
EXT = .exe
42+
endif
43+
44+
build: version.py $(PROGRAMS) man CITATION.cff about.md
45+
46+
version.py: .FORCE
47+
cmt codemeta.json version.py
48+
49+
hash: .FORCE
50+
git log --pretty=format:'%h' -n 1
51+
52+
man: $(MAN_PAGES_1) # $(MAN_PAGES_3) $(MAN_PAGES_7)
53+
54+
$(MAN_PAGES_1): .FORCE
55+
mkdir -p man/man1
56+
pandoc $@.md --from markdown --to man -s >man/man1/$@
57+
58+
$(MAN_PAGES_3): .FORCE
59+
mkdir -p man/man3
60+
pandoc $@.md --from markdown --to man -s >man/man3/$@
61+
62+
$(MAN_PAGES_7): .FORCE
63+
mkdir -p man/man7
64+
pandoc $@.md --from markdown --to man -s >man/man7/$@
65+
66+
$(PROGRAMS): $(PACKAGE)
67+
@mkdir -p bin
68+
go build -o "bin/$@$(EXT)" cmd/$@/*.go
69+
@./bin/$@ -help >$@.1.md
70+
71+
$(MAN_PAGES): .FORCE
72+
mkdir -p man/man1
73+
pandoc $@.md --from markdown --to man -s >man/man1/$@
74+
75+
CITATION.cff: codemeta.json
76+
cmt codemeta.json CITATION.cff
77+
78+
about.md: codemeta.json $(PROGRAMS)
79+
cmt codemeta.json about.md
80+
81+
test: $(PACKAGE)
82+
uv run test_dataset.py
83+
84+
website: clean-website .FORCE
85+
make -f website.mak
86+
87+
status:
88+
git status
89+
90+
save:
91+
@if [ "$(msg)" != "" ]; then git commit -am "$(msg)"; else git commit -am "Quick Save"; fi
92+
git push origin $(BRANCH)
93+
94+
refresh:
95+
git fetch origin
96+
git pull origin $(BRANCH)
97+
98+
publish: build website .FORCE
99+
./publish.bash
100+
101+
clean:
102+
@if [ -f version.go ]; then rm version.go; fi
103+
@if [ -d bin ]; then rm -fR bin; fi
104+
@if [ -d dist ]; then rm -fR dist; fi
105+
@if [ -d man ]; then rm -fR man; fi
106+
@if [ -d testout ]; then rm -fR testout; fi
107+
108+
clean-website:
109+
@for FNAME in $(HTML_PAGES); do if [ -f "$${FNAME}" ]; then rm "$${FNAME}"; fi; done
110+
111+
112+
.FORCE:

‎README.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11

2-
# py_dataset [![DOI](https://data.caltech.edu/badge/175684474.svg)](https://data.caltech.edu/badge/latestdoi/175684474)
2+
[![DOI](https://data.caltech.edu/badge/175684474.svg)](https://data.caltech.edu/badge/latestdoi/175684474)
33

4-
py_dataset is a Python wrapper for the [dataset](https://github.com/caltechlibrary/dataset)
5-
command line implementation. It replaces the depreciated libdataset a C shared library starting
4+
# py_dataset
5+
6+
py_dataset is a Python wrapper for the [dataset](https://github.com/caltechlibrary/dataset)
7+
command line tools. It replaces the depreciated libdataset a C shared library starting
68
with the dataset 2.2.x release.
79

810
This package wraps all [dataset](docs/) operations such
@@ -110,6 +112,10 @@ Then run the following Python commands.
110112
# read returns a touple so we're printing the first one.
111113
print(dataset.read(c_name, key)[0])
112114

115+
# Now let's query the collection.
116+
sql_stmt = f'''select src from {c_name} order by created desc'''
117+
print(dataset.query(c_name, sql_stmt))
118+
113119
# Finally we can remove (delete) a record from our collection
114120
if not dataset.delete(c_name, key):
115121
print(dataset.error_message())

‎about.html‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
<!DOCTYPE html>
2+
<html lang="en-US">
3+
<head>
4+
<title>py_dataset</title>
5+
<link rel="stylesheet" href="/css/site.css">
6+
</head>
7+
<body>
8+
<nav>
9+
<ul>
10+
<li><a href="/">Home</a></li>
11+
<li><a href="index.html">README</a></li>
12+
<li><a href="LICENSE">LICENSE</a></li>
13+
<li><a href="user_manual.html">User Manual</a></li>
14+
<li><a href="about.html">About</a></li>
15+
<!-- <li><a href="search.html">Search</a></li> -->
16+
<li><a href="https://github.com/caltechlibrary/py_dataset">GitHub</a></li>
17+
</ul>
18+
</nav>
19+
<section>
20+
<h1 id="about-this-software">About this software</h1>
21+
<h2 id="py_dataset-2.2.3.1">py_dataset 2.2.3.1</h2>
22+
<p>This patch adds missing dsquery support.</p>
23+
<h3 id="authors">Authors</h3>
24+
<ul>
25+
<li>Robert Doiel, <a href="https://orcid.org/0000-0003-0900-6903"
26+
class="uri">https://orcid.org/0000-0003-0900-6903</a></li>
27+
<li>Thomas E Morrell, <a href="https://orcid.org/0000-0001-9266-5146"
28+
class="uri">https://orcid.org/0000-0001-9266-5146</a></li>
29+
</ul>
30+
<h3 id="maintainers">Maintainers</h3>
31+
<ul>
32+
<li>R. S. Doiel, <a href="https://orcid.org/0000-0003-0900-6903"
33+
class="uri">https://orcid.org/0000-0003-0900-6903</a></li>
34+
<li>Thomas E Morrell, <a href="https://orcid.org/0000-0001-9266-5146"
35+
class="uri">https://orcid.org/0000-0001-9266-5146</a></li>
36+
</ul>
37+
<p>A command line tool for working with JSON documents on local disc</p>
38+
<ul>
39+
<li>License: <a
40+
href="https://github.com/caltechlibrary/py_dataset/blob/main/LICENSE"
41+
class="uri">https://github.com/caltechlibrary/py_dataset/blob/main/LICENSE</a></li>
42+
<li>GitHub: <a href="https://github.com/caltechlibrary/py_dataset"
43+
class="uri">https://github.com/caltechlibrary/py_dataset</a></li>
44+
<li>Issues: <a
45+
href="https://github.com/caltechlibrary/py_dataset/issues"
46+
class="uri">https://github.com/caltechlibrary/py_dataset/issues</a></li>
47+
</ul>
48+
<h3 id="programming-languages">Programming languages</h3>
49+
<ul>
50+
<li>Python3</li>
51+
</ul>
52+
<h3 id="software-requirements">Software Requirements</h3>
53+
<ul>
54+
<li>dataset &gt;= 2.2.3</li>
55+
</ul>
56+
</section>
57+
</body>
58+
</html>

‎about.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ maintainer:
1919
id: https://orcid.org/0000-0001-9266-5146
2020

2121
repository_code: https://github.com/caltechlibrary/py_dataset
22-
version: 2.2.2
22+
version: 2.2.3.1
2323
license_url: https://github.com/caltechlibrary/py_dataset/blob/main/LICENSE
2424

2525
programming_language:
@@ -32,15 +32,15 @@ keywords:
3232
- software
3333
- json
3434

35-
35+
date_released: 2025-04-17
3636
---
3737

3838
About this software
3939
===================
4040

41-
## py_dataset 2.2.2
41+
## py_dataset 2.2.3.1
4242

43-
This release of py_dataset drops the support for the old C shared libraries available before v2.2.0. It implements a wrapper around the dataset command to perform a similar function.
43+
This patch adds missing dsquery support.
4444

4545
### Authors
4646

@@ -71,5 +71,5 @@ A command line tool for working with JSON documents on local disc
7171

7272
### Software Requirements
7373

74-
- dataset &gt;&#x3D; 2.2.2
74+
- dataset &gt;&#x3D; 2.2.3
7575

0 commit comments

Comments
 (0)