Skip to content

Lower bandwidth at startup #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 17, 2019
Merged

Lower bandwidth at startup #264

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
783906c
Use gzipped version of clojars artifacts
paulrd Aug 1, 2019
c30a682
Write artifacts to local file
paulrd Aug 1, 2019
5e0cba6
Use .artifacts-cache file to cache artifacts between runs
paulrd Aug 1, 2019
34f83cf
check if cache file exists
paulrd Aug 1, 2019
1ec5101
moved cache file to system temp director
paulrd Aug 1, 2019
9b2166b
Update changelog for PR #264
paulrd Aug 1, 2019
81339ba
keep map of artifacts sorted after reading back from disk
paulrd Aug 1, 2019
66ffc0a
Read from disk as edn data (subset of clojure is safer)
paulrd Aug 2, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ install.cmd
/.cljs_node_repl/
/out/
/.lein-env
.inline-deps
.inline-deps
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

* [#251](https://github.com/clojure-emacs/refactor-nrepl/pull/251) `clean-ns` support extra message key `relative-path`, which will be used if `path` does not exist.
* [#256](https://github.com/clojure-emacs/refactor-nrepl/pull/256) ignore malformed artifact coordinates when fetching from Clojars.
* [#264](https://github.com/clojure-emacs/refactor-nrepl/pull/264) less bandwidth used to fetch artifacts from Clojars

## 2.4.0

Expand Down
27 changes: 22 additions & 5 deletions src/refactor_nrepl/artifacts.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,24 @@
[refactor-nrepl.ns.slam.hound.regrow :as slamhound-regrow]
[version-clj.core :as versions])
(:import java.util.Date
java.util.zip.GZIPInputStream
java.util.jar.JarFile))

(def artifacts-file (str (System/getProperty "java.io.tmpdir")
"/refactor-nrepl-artifacts-cache"))

(defn get-last-modified-from-file
"Returns last modified time in milliseconds or nil if file does not exist."
[file]
(let [lm (.lastModified (io/file file))]
(if (zero? lm) nil lm)))

;; structure here is {"prismatic/schem" ["0.1.1" "0.2.0" ...]}
(defonce artifacts (atom {} :meta {:last-modified nil}))
(defonce artifacts (atom (if (.exists (io/as-file artifacts-file))
(->> artifacts-file slurp edn/read-string (into (sorted-map)))
{})
:meta {:last-modified
(get-last-modified-from-file artifacts-file)}))
(def millis-per-day (* 24 60 60 1000))

(defn- get-proxy-opts
Expand All @@ -27,7 +41,7 @@
(defn- stale-cache?
[]
(or (empty? @artifacts)
(if-let [last-modified (some-> artifacts meta :last-modified .getTime)]
(if-let [last-modified (some-> artifacts meta :last-modified)]
(neg? (- millis-per-day (- (.getTime (java.util.Date.)) last-modified)))
true)))

Expand All @@ -43,8 +57,9 @@
"Returns a vector of [[some/lib \"0.1\"]...]."
[]
(try
(->> "https://clojars.org/repo/all-jars.clj"
java.net.URL.
(->> "https://clojars.org/repo/all-jars.clj.gz"
io/input-stream
GZIPInputStream.
io/reader
line-seq
(keep edn-read-or-nil))
Expand Down Expand Up @@ -97,7 +112,9 @@
(let [clojars-artifacts (future (get-artifacts-from-clojars!))
maven-artifacts (future (get-artifacts-from-mvn-central!))]
(reset! artifacts (into @clojars-artifacts @maven-artifacts))
(alter-meta! artifacts update-in [:last-modified] (constantly (java.util.Date.)))))
(spit artifacts-file @artifacts)
(alter-meta! artifacts update-in [:last-modified]
(constantly (get-last-modified-from-file artifacts-file)))))

(defn artifact-list
[{:keys [force]}]
Expand Down