Integrated uploader #2

PaulZC · 2020-06-09T10:04:50Z

Hi @oclyke / @nseidle ,
I think I'm about done with this. I've updated the README and added the pyinstaller build instructions. It is missing the executables for OSX and Linux, please drop those in when time permits (and delete the DELETE_ME.txt placeholders).
The code is not quite as elegant as I would like - it passes many things using self. 'globals' - Owen may need to put on his rose tinted specs when reviewing it! - but it is a neat stand-alone arg-less one-file solution.
Enjoy!
Paul

…tings is None)

nseidle · 2020-06-10T21:47:26Z

I'm testing on windows and a few oddities:

I like all of the graphic improvements (better button placement, bold, etc).
I like the universal blink
Did you pyinstaller with --noconsole? The gui.exe opens a console window when I don't think it needs to.
It takes ~6 seconds of blank console before the GUI loads on my fairly powerful desktop. Recompiling with --noconsole, the console goes away, but the load time is the same. Since there is no console, I get a spinning wheel while the app loads. This seems to be acceptable to me as a user (and I suspect this massive python script is just going to take that long to load!). Please use --noconsole for at least the windows version and we should be good.
Update Bootloader fails for me every time on a RedBoard and on an OLA. Output below. No smoking gun, just seems to fail to get the module into BL mode. DTR problem? Trying again on v1.0, it takes a few tries but I can eventually get the bootloader to update.

Switching back to v2.0, hitting 'update bootloader' goes very fast and immediately fails. I don't think 'Tries = 3' is getting reset to 0? Yep, I fixed this and committed. I still can't bootload a new BL.

  Artemis Bootloader Update
  Installing bootloader version 5
  Header Size = 0x80
  original app_size 12944
  load_address 0xc000
  app_size 12944
  w0 = 0xcb003310
  Security Value 0x10
  w2 = 0x10008080
  addrWord = 0xc000
  versionKeyWord = 0x0
  child0/feature = 0xffffffff
  child1 = 0xffffffff
  crc =  0x826a7429
  Writing to file application_OTA_blob.bin
  testing: application_OTA_blob.bin
  Header Size = 0x60
  app_size = 13072
  Writing to file application_Wired_OTA_blob.bin
  Image from 0x0 to 0x3310 will be loaded at 0x20000
  Connecting over serial port...
  Sending Hello.
  No response for command bytearray(b'\x00\x00\x08\x00')
  Failed to respond
  Fail
  Sending Hello.
  No response for command bytearray(b'\x00\x00\x08\x00')
  Failed to respond
  Fail
  Sending Hello.
  No response for command bytearray(b'\x00\x00\x08\x00')
  Failed to respond
  Fail
  Tries = 3
  Upload failed!

Please get Update Bootloader working then I'm ready to merge.

PaulZC · 2020-06-11T09:39:47Z

Thanks @nseidle ,
Thanks for the --noconsole suggestion. Done.
I'm guessing/hoping your Update Bootloader problems were baud rate related.
On my Windows machine I can upload at 921600 but to update the bootloader I have to drop the rate to 115200.
On RPi I can only upload at 460800. 921600 fails.
So, I've done two things:

Update Bootloader now uses 115200 (whether you like it or not)
If the Upload fails and the baud rate is set higher than 115200, you get a helpful "Please try a lower Baud Rate" message

Hopefully all better now?
Paul

nseidle · 2020-06-11T15:38:22Z

All better! Thanks.

I forgot about the baud for uploading. Yep, that was the issue. Nice catch and fix.

I really it. @oclyke Please have a review so we can merge at your convenience.

oclyke · 2020-06-11T16:25:53Z

Update Bootloader now uses 115200 (whether you like it or not)

This is probably OK considering that this is the "Artemis" firmware uploader. However I think it is worth mentioning that the Edge board is still produced with the ASB set to 921600. Could/should there be a small menu option to enable use of this tool with the Edge?

oclyke

Changes look good to me. I'll now take it for a test drive and also generate Mac / Linux binaries

PaulZC · 2020-06-11T16:42:07Z

Thanks @oclyke !
I don't have any hardware to test it on, but adding a checkbox for the Edge (that forces Update Bootloader to use 921600) is no problem. Just say the word and I'll add it.
All the best,
Paul

oclyke · 2020-06-11T16:44:59Z

I'm onboard.... @nseidle do you think we should support the Edge here? THe drawback is possibly clutter in UI but I can't think of anything else. I will hold off on generating binaries until we decide which way to go.

nseidle · 2020-06-11T16:46:32Z

Agreed, add the checkbox. This is a tool, I'd rather have functionality than form.

PaulZC · 2020-06-11T16:51:09Z

OK. Leave it with me. I'll try and rotate the label so it runs vertically up the left edge (geddit?!) of the GUI. Unless that's a lame idea?!

oclyke · 2020-06-11T17:20:21Z

Clever idea - I'd like to see it at least to see if it is useable - worried it could be hard to read.

In the meantime I just got to testing this on my Mac. No matter what serial port I select I get an immediate failure with this message:

Artemis SVL Uploader
Could not communicate with board!

Also of note (I guess) is that hitting the button again did not produce any more of that error message. I could not tell if it was trying again and getting the same error and overwriting it in the status window, or if it was just not trying after the one error.

Some thoughts:

somehow reduce confusion about error messages - maybe show a blank status window for 0.25 sec after hitting a button
maybe add a way for users to manually clear the status console?
my serial port selection was lost when clicking 'refresh' - not the biggest problem but could be a nice feature to persist that when the selected port is still available after refresh

Changed RemoteWidget(QWidget) to MainWindow(QMainWindow) to make it easier to add the Board Type menu and status bar. The QWidget labels and buttons did not need to be self.'globals'. Removed the self. from those. Added the Board Type menu with checkboxes for Artemis (for Artemis-based boards) and Apollo3 (for the Edge). Added statusTip's for each. Added SETTINGS_ARTEMIS. Refresh (update_com_ports) now restores the previous port if it still exists. Commented out messages.clear() so previous messages are retained. Update bootloader baud rate is set automatically to 115200 for Artemis boards and 921600 for Apollo3 boards.

PaulZC · 2020-06-12T10:33:58Z

Hi @oclyke / @nseidle ,

I've made a few changes and we now have a Board Type menu with radio buttons for Aretmis and Apollo3 (Edge). The status bar at the bottom of the window displays helpful information when you hover over the two options. Update Bootloader automatically uses 115200 for Artemis and 921600 for Apollo3.

I've stopped the messages box from being cleared between updates/uploads, so you can see the full history.

Refreshing the com ports will leave the previous port selected (if it still exists).

A nice touch is that the window size can now be changed too - by dragging the bottom right corner of the window.

I can't explain why you're seeing Mac com failures but I suspect it is something fairly low level linked to the serial port and/or its baud rate capability. On my Windows machine, I see an instant failure if I leave COM1 selected (the default non-USB com port) and have 921600 baud selected. If I change the baud rate to 115200, the code does attempt to talk to COM1 before timing out.

If you have time, can you please try running the artemis_svl.py and ambiq_bin2board.py as per this post. Does that work?

Adam Garbo has built and run the executable successfully on macOS Catalina 10.15.4. Here's a copy of his pip3 list just in case that helps? This was before he replaced crypto with pycryptodome (pip3 uninstall crypto followed by pip3 install pycryptodome)

All the best,
Paul

oclyke · 2020-06-15T18:46:13Z

Here's how I tested:

Updated SVL via Arduino (known good method)
Tried to upload a binary via command line tools included with Arduino (the Python version of the executables that are used by Arduino) Result:

owenlyke@Owens-MacBook-Pro-10 1.1.1 % python3 tools/artemis/artemis_svl.py -v -f ~/Desktop/Temporary/Blink/Blink.ino.bin /dev/cu.wchusbserial1410


Artemis SVL Bootloader
Script version 1.7

Phase:	Setup
	Cleared startup blip
	Got SVL Bootloader Version: 5
	Sending 'enter bootloader' command

Phase:	Bootload
	have 8660 bytes to send in 5 frames
	Sending frame #1, length: 2048
	Sending frame #2, length: 2048
	Sending frame #3, length: 2048
	Sending frame #4, length: 2048
	Sending frame #5, length: 468

	Upload complete

	Nominal bootload bps: 8870.31

Fetch latest changes from your PR in the GUI repo
a. test a few of your feature additions (nice!)
Update bootloader using GUI (success)

Updating bootloader

Artemis Bootloader Update

Installing bootloader version 5
Header Size = 0x80
original app_size 12944
load_address 0xc000
app_size 12944
w0 = 0xcb003310
Security Value 0x10
w2 = 0x10008080
addrWord = 0xc000
versionKeyWord = 0x0
child0/feature = 0xffffffff
child1 = 0xffffffff
crc =  0x826a7429
Writing to file application_OTA_blob.bin
testing: application_OTA_blob.bin
Header Size = 0x60
app_size = 13072
Writing to file application_Wired_OTA_blob.bin
Image from 0x0 to 0x3310 will be loaded at 0x20000
Connecting over serial port...
Changing Baud Rate to 115200
Sending Hello.
Received response for Hello
Bootloader connected
Received Status
Length = 0x58
Version = 0x5
Max Storage = 0x4ffa0
Status = 0x2
State = 0x7
Sending OTA Descriptor = 0xfe000
Sending Update Command.
Number of updates needed = 1
Sending block of size 0x3370 from 0x0 to 0x3370
Sending Data Packet of length 8180
Sending Data Packet of length 4988
Sending Reset Command.
Tries = 0
Bootloader updated to version 5

Test uploading code at 921600 baud (success)

Uploading firmware

Artemis SVL Uploader

Phase:	Setup
	Cleared startup blip
	Got SVL Bootloader Version: 5
	Sending 'enter bootloader' command
Phase:	Bootload
	Sending 8784 bytes in 5 frames
	Got frame request
	Sending frame #1, length: 2048
	Got frame request
	Sending frame #2, length: 2048
	Got frame request
	Sending frame #3, length: 2048
	Got frame request
	Sending frame #4, length: 2048
	Retrying...
	Sending frame #4, length: 2048
	Got frame request
	Sending frame #5, length: 592
	Got frame request
Upload complete!
Nominal bootload 6097.96 bytes/sec

So somewhere in there my issues got resolved, it seems. I will go ahead and generate binaries for mac / linux (ubuntu).

Thanks Paul, this is sweet!

generated with: owenlyke@Owens-MacBook-Pro-10 tools % pyinstaller --onefile --noconsole --distpath=. --icon=artemis_firmware_uploader_gui.ico --add-data="artemis_svl.bin:." --add-data="Artemis-Logo-Rounded.png:." artemis_firmware_uploader_gui.py

generated with pyinstaller --onefile --noconsole --distpath=. --icon=artemis_firmware_uploader_gui.ico --add-data="artemis_svl.bin:." --add-data="Artemis-Logo-Rounded.png:." artemis_firmware_uploader_gui.py

oclyke · 2020-06-15T20:01:48Z

@PaulZC I've just added the binaries for OSX and Ubuntu. Nate and I agree that you can go ahead and merge this PR 👍

PaulZC · 2020-06-15T21:11:35Z

Thanks @oclyke - glad you like it!
Can you or @nseidle please update the screenshot when you have time?
https://cdn.sparkfun.com/assets/home_page_posts/3/2/4/5/Artemis_Firmware_Uploader_GUI.jpg
Cheers!

nseidle · 2020-06-15T21:18:46Z

@PaulZC New image here:

https://cdn.sparkfun.com/assets/home_page_posts/3/1/2/3/Artemis-GUI-v2.jpg

PaulZC added 16 commits June 3, 2020 13:45

Work in progress

310419f

Work in progress - uploader almost works

df80c50

Upload works

a9e084f

Updater - work in progress

c3b5e6b

Updater is almost working - still needs more work

7500dae

Working nicely on Windows!

a7d1d1f

Working nicely on Windows - needs to be tested on Linux/OSX

c47d948

Code now prompts if the bootloader needs to be updated

3cb7d41

Moving the .exe up one directory. Deleting the redundant .exe's.

6fd518e

Change the order of the baud rates (so code defaults to 115200 if set…

79a0ee7

…tings is None)

Adding the Raspberry Pi executable

f2a2468

Updated Blink.bin so it will work on all the boards

8300545

Create LICENSE.md

55fe052

Update README.md

de930af

Updated the README. Added placeholders for Linux and OSX.

6e09559

Update README.md

019dd1f

PaulZC requested a review from oclyke June 9, 2020 10:04

Reset loadTries between bootload attempts

4cd1a7b

PaulZC added 2 commits June 11, 2020 08:04

Changed baud rate to 115200 for update bootloader. Added --noconsole.

261bf8d

Added the Baud Rate warning. Updated the Windows and RPi executables.

f80d8b1

oclyke approved these changes Jun 11, 2020

View reviewed changes

PaulZC added 4 commits June 12, 2020 10:15

Updating the Windows .exe

f8339be

Update README.md

c8dbf5a

Update the RPi executable

4ee0f4d

oclyke added 2 commits June 15, 2020 12:54

add compiled binary for OSX

0d51abe

generated with: owenlyke@Owens-MacBook-Pro-10 tools % pyinstaller --onefile --noconsole --distpath=. --icon=artemis_firmware_uploader_gui.ico --add-data="artemis_svl.bin:." --add-data="Artemis-Logo-Rounded.png:." artemis_firmware_uploader_gui.py

add compiled ubuntu binary

67f31a2

generated with pyinstaller --onefile --noconsole --distpath=. --icon=artemis_firmware_uploader_gui.ico --add-data="artemis_svl.bin:." --add-data="Artemis-Logo-Rounded.png:." artemis_firmware_uploader_gui.py

PaulZC merged commit fe44f88 into master Jun 15, 2020

PaulZC deleted the PaulZC_Integrated_Uploader branch June 15, 2020 20:51

PaulZC mentioned this pull request Jun 15, 2020

Mac / Linux Support #1

Closed

Integrated uploader #2

Integrated uploader #2

Uh oh!

Conversation

PaulZC commented Jun 9, 2020

Uh oh!

nseidle commented Jun 10, 2020

Uh oh!

PaulZC commented Jun 11, 2020

Uh oh!

nseidle commented Jun 11, 2020

Uh oh!

oclyke commented Jun 11, 2020

Uh oh!

oclyke left a comment

Choose a reason for hiding this comment

Uh oh!

PaulZC commented Jun 11, 2020

Uh oh!

oclyke commented Jun 11, 2020

Uh oh!

nseidle commented Jun 11, 2020

Uh oh!

PaulZC commented Jun 11, 2020

Uh oh!

oclyke commented Jun 11, 2020

Uh oh!

PaulZC commented Jun 12, 2020

Uh oh!

oclyke commented Jun 15, 2020

Uh oh!

oclyke commented Jun 15, 2020

Uh oh!

PaulZC commented Jun 15, 2020

Uh oh!

nseidle commented Jun 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants