Allow listing other boards in boards manager #3055

ffissore · 2015-04-29T13:19:52Z

This PR is a work in progress. Once completed, the IDE built by @ArduinoBot will be able to download additional package_SOMETHING_index.json files and list their contents in Boards Manager

Additional info on dev mailing list: https://groups.google.com/a/arduino.cc/d/msg/developers/w2tOTjh6F5I/Y3Eexp4qeb4J

ffissore · 2015-04-29T13:20:11Z

This first commit adds support for additional package_SOMETHING_index.json files, found at the same location of package_index.json

ffissore · 2015-04-30T12:37:02Z

With the last build, you can:

close the IDE
edit preferences.txt file
add an entry boardsmanager.additional.urls=http://someurl...
reopen the IDE and open boards manager
and see your custom board(s) listed

q2dg · 2015-04-30T17:33:29Z

The final dev info should be place in wiki, shouldn't it?

toddtreece · 2015-04-30T17:40:48Z

@ffissore HTTP custom package index files work great for me on OS X 10.10.3, but HTTPS fails with this error:

 Error downloading https://apt.adafruit.com/proxy_files/package_adafruit_index.json
java.lang.RuntimeException: java.lang.Exception: Error downloading https://apt.adafruit.com/proxy_files/package_adafruit_index.json
    at cc.arduino.contributions.packages.ui.ContributionManagerUI$3.run(ContributionManagerUI.java:155)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Error downloading https://apt.adafruit.com/proxy_files/package_adafruit_index.json
    at cc.arduino.contributions.packages.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:99)
    at cc.arduino.contributions.packages.ContributionInstaller.downloadIndex(ContributionInstaller.java:276)
    at cc.arduino.contributions.packages.ContributionInstaller.downloadIndexAndSignature(ContributionInstaller.java:252)
    at cc.arduino.contributions.packages.ContributionInstaller.updateIndex(ContributionInstaller.java:243)
    at cc.arduino.contributions.packages.ui.ContributionManagerUI$3.run(ContributionManagerUI.java:151)
    ... 1 more
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2011)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1113)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1375)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
    at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:158)
    at cc.arduino.contributions.packages.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:97)
    ... 5 more

The server it's being hosted on doesn't seem to have any SSL issues:

I'm using this file to test, but we can just point people to HTTP if you don't plan to support HTTPS right now: https://apt.adafruit.com/proxy_files/package_adafruit_index.json

ffissore · 2015-04-30T18:30:03Z

@q2dg of course, once merged. Would you like to start working on it?

ffissore · 2015-04-30T18:31:35Z

@toddtreece ahh ssl stuff, I guess it's my fault. I'll fix it this Monday (vacation weekend)

ffissore · 2015-04-30T18:37:00Z

@toddtreece I see that url works also with plain http. Can you use it to test the rest?

ffissore · 2015-05-04T09:14:46Z

@toddtreece I dug into that error and I'm sorry to say it's a cipher suite issue: your server is requesting an encryption strength that is subject to US encryption algorithms export limitations.
The standard JVM (the one we bundle with Windows and Mac) misses some encryption algorithms.
Indeed, scrolling down to "handshake simulation" section, you'll see that JVM 6, 7 and 8 can't connect.

Three fixes available:

you can manually download and install Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files
We really can't require that: it will make the installation flow impossible.
you can change your webserver configuration so that it supports weaker algorithms too. I can't really ask you that though.
you can provide a plain http url: this one is the easiest as it already works

…ge_SOMETHING_index.json

…x file

…build time

…ce 'boardsmanager.additional.urls'

…age_" are preserved

…es under "Contributed" category

…ed at build time even when building a "light bundle"

ffissore · 2015-05-04T12:20:16Z

With 71106ed it's now possible to specify additional urls from File > Preferences. They have to be comma separated

toddtreece · 2015-05-04T13:30:10Z

@ffissore good catch. i'll see if i can configure nginx to work with java without exposing the server to attacks, but HTTP will work fine for now.

The File > Preferences addition works great for me on OS X 10.10.3, but the IDE is reporting this for the main Arduino package index: http://downloads.arduino.cc/packages/package_index.json file signature verification failed. File ignored

ffissore · 2015-05-04T13:36:08Z

Have you disabled your proxy? I can't think of anything else atm

toddtreece · 2015-05-04T13:40:41Z

@ffissore that was the issue. thanks. everything is looking good here on OS X.

…ere preventing Boards Manager from working on Windows

ArduinoBot · 2015-05-04T14:30:24Z

Build successful. Please test this code using one of the following:
http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-3055-BUILD-262-linux32.tar.xz
http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-3055-BUILD-262-linux64.tar.xz
http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-3055-BUILD-262-windows.zip
http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-3055-BUILD-262-macosx.zip

Allow listing other boards in boards manager

Lauszus · 2015-05-04T20:11:19Z

Thanks for working on this @ffissore!

I just tried it with the following json file: http://downloads.balanduino.net/arduino_package/package_tkj_balanduino_index.json and it works fine.

However I am getting the following exception:

The full exception is here:

java.io.IOException: invalid header encountered
    at org.bouncycastle.bcpg.BCPGInputStream.readPacket(Unknown Source)
    at org.bouncycastle.openpgp.PGPSignature.<init>(Unknown Source)
    at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(Unknown Source)
    at cc.arduino.contributions.GPGDetachedSignatureVerifier.verify(GPGDetachedSignatureVerifier.java:61)
    at cc.arduino.contributions.packages.ContributionsIndexer.isSigned(ContributionsIndexer.java:153)
    at cc.arduino.contributions.packages.ContributionsIndexer.mergeContributions(ContributionsIndexer.java:100)
    at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:82)
    at processing.app.BaseNoGui.initPackages(BaseNoGui.java:600)
    at processing.app.Base.openInstallBoardDialog(Base.java:1285)
    at processing.app.Base.access$100(Base.java:74)
    at processing.app.Base$11.actionPerformed(Base.java:1301)
    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2346)
    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
    at javax.swing.AbstractButton.doClick(AbstractButton.java:376)
    at com.apple.laf.ScreenMenuItem.actionPerformed(ScreenMenuItem.java:125)
    at java.awt.MenuItem.processActionEvent(MenuItem.java:669)
    at java.awt.MenuItem.processEvent(MenuItem.java:628)
    at java.awt.MenuComponent.dispatchEventImpl(MenuComponent.java:351)
    at java.awt.MenuComponent.dispatchEvent(MenuComponent.java:339)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:754)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:702)
    at java.awt.EventQueue$3.run(EventQueue.java:696)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.awt.EventQueue$4.run(EventQueue.java:724)
    at java.awt.EventQueue$4.run(EventQueue.java:722)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:721)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Furthermore it would be really nice if one could specify library dependencies, just as one can specify tools dependencies.

In my specific case I would like to add: https://github.com/TKJElectronics/KalmanFilter and https://github.com/felis/USB_Host_Shield_2.0 which are both found in the library manager.

Do you want me to open up a separate issue for it?

Lauszus · 2015-05-04T23:11:28Z

I got it working by hosting it on Github instead: https://raw.githubusercontent.com/TKJElectronics/Balanduino/master/package_tkj_balanduino_index.json :)

ffissore · 2015-05-05T06:51:00Z

Thank you @Lauszus for those two error messages: we now handle those two conditions 060ec30
As for library dependencies, we don't support it yet, so yes please open a separate issue

Lauszus · 2015-05-05T13:14:50Z

Thanks @ffissore. I just opened up a new issue: #3082.

ffissore mentioned this pull request Apr 30, 2015

Allow board manager to install third-party hardware add-ons #2949

Closed

ffissore added the Component: Board/Lib Manager Boards Manager or Library Manager label May 1, 2015

ffissore mentioned this pull request May 4, 2015

Published URL no longer activates unrelated features like per job messages and PR autoclose jenkinsci/ghprb-plugin#68

Closed

Federico Fissore added 11 commits May 4, 2015 12:05

ContributionIndexer now loads and merges content of files named packa…

804480c

…ge_SOMETHING_index.json

Introducing GPG signature verification when loading main package_inde…

cf1879f

…x file

Copying bundled signature file if missing from user's preferences folder

1f1f5b5

Bundled package and library index: downloading gzipped json files at …

0292d8b

…build time

Boards Manager: downloading additional files as specified in preferen…

ecf432f

…ce 'boardsmanager.additional.urls'

Boards Manager: index was updated twice. Fixed

9d65b41

Boards Manager: package index file with name starting with "test_pack…

cddfa28

…age_" are preserved

Boards Manager: platforms listed in non-signed package index files go…

a214564

…es under "Contributed" category

Boards/Library Managers: package and library index files get download…

2b2c978

…ed at build time even when building a "light bundle"

ContributionInstaller: downloadIndex -> download

19bb40c

Boards Manager: additional URLs may be specified from File > Preferences

71106ed

Lots of unclosed input and output streams now properly closed. They w…

cd49d29

…ere preventing Boards Manager from working on Windows

ffissore added a commit that referenced this pull request May 4, 2015

Merge pull request #3055 from ffissore/other-packages

87fb45c

Allow listing other boards in boards manager

ffissore merged commit 87fb45c into arduino:master May 4, 2015

ffissore deleted the other-packages branch May 4, 2015 14:55

Lauszus mentioned this pull request May 5, 2015

Library dependencies for boards added via the boards manager #3082

Open

ffissore modified the milestone: Release 1.6.5 May 20, 2015

Uh oh!

Allow listing other boards in boards manager #3055

Allow listing other boards in boards manager #3055

Uh oh!

Conversation

ffissore commented Apr 29, 2015

Uh oh!

ffissore commented Apr 29, 2015

Uh oh!

ffissore commented Apr 30, 2015

Uh oh!

q2dg commented Apr 30, 2015

Uh oh!

toddtreece commented Apr 30, 2015

Uh oh!

ffissore commented Apr 30, 2015

Uh oh!

ffissore commented Apr 30, 2015

Uh oh!

ffissore commented Apr 30, 2015

Uh oh!

ffissore commented May 4, 2015

Uh oh!

ffissore commented May 4, 2015

Uh oh!

toddtreece commented May 4, 2015

Uh oh!

ffissore commented May 4, 2015

Uh oh!

toddtreece commented May 4, 2015

Uh oh!

ArduinoBot commented May 4, 2015

Uh oh!

Lauszus commented May 4, 2015

Uh oh!

Lauszus commented May 4, 2015

Uh oh!

ffissore commented May 5, 2015

Uh oh!

Lauszus commented May 5, 2015

Uh oh!

Uh oh!