Different smart screen implementation in newer models?

[gerph]

Describe the bug

The screen that I bought appears to have a different protocol, which isn't supported by this code - it has a separately controllable backlight (actually 4 lights, but they seem to all work the same way) and uses a 10 byte control sequence, rather than the 6 byte sequence used by the older models.

To Reproduce
Steps to reproduce the behavior:

1. Buy the 'flagship' model from https://www.aliexpress.com/item/1005003723773653.html (probably others, but that's where I got mine)
2. Run this software and see nothing happens.
3. Be sad.
4. Snoop the serial protocol on a Windows machine.
5. Update the code to have the new protocol.
6. Ask on this bug whether it's worth offering a fork with a PR in?

Expected behavior
I was excited you'd done the protocol analysis, and sad when I bought a different device and it didn't work. But now I have something that does work!

Screenshots / photos of the Turing screen
This model:

Backlight is visible on the front and more obviously on the rear.

Environment:

• Working with master + changes for new model
• OS with version OSX 11.5.2
• Python version 3.9.8
• Hardware MacBook Pro.

Additional context

The new product has a different format for its commands...

• Command packets are 10 bytes long, framed by the command byte at either end.
• Command bytes are 0xCA-0xCE.
• A 'HELLO' command elicits a response from the device of 'HELLO' followed by 0x0A, 0x12, 0x00. Useful to identify the device format.
• An 'orientation' packet allows the orientation of the next bitmap block to be selected from landscape and portrait (column major or row major).
• The graphics data is still 565, BUT... high 3 bits of green are in b0-b2, low 3 bits of green are in b13-15, red is in b3-7, blue is in b8-12.
• A 'lighting' packet lets you select the colours of the backlight LEDs with 8bit RGB values.
• Amusingly the 'lighting' isn't supported by the software that the seller included in their listing - although they mention it, there's no way to set the colours in their software.
• There is no Clear, Screen off, or Screen on packet (as far as I can tell).

The device has a serial number of '2017-2-25' - so whilst this appears to be newer, the year implies that it was created longer ago?

Would you like a PR of the results of my investigation which works with the device?

[mathoudebine]

Hi @gerph!

Thanks for submitting this issue, this is the first time I hear about a different Turing Smart Screen model! Seems the packaging has been reworked because on the first model we could see and access some electronic components from the backplate.

I will order this new model right away, meanwhile I am very interested by a PR with your changes so that it can be included in this repository! I think more and more people will order this new version and would be interested as well.

Can you also send me the link where to download the seller Windows app for this new version? I would be curious to decompile it the same way I did with the first version app, and find maybe some hidden commands.

Thanks again for your work and your investigations!

[gerph]

The software that worked for the different model is http://xf.allinking.com/XFZX/XFZX.zip?spm=a2g0o.detail.1000023.13.3e906bd1MOu2iC&file=XFZX.zip - it was linked in the listing, together with what looked like the older one with the USBMonitor in. There are pictures in the listing to show how to change it to English - I cannot read chinese so that was important to me!

I decoded the protocol using https://www.aggsoft.com/serial-port-monitor.htm trial (which only allows 10K to be sniffed, but that was enough) and later using https://freeserialanalyzer.com/ when I had some kernel crashes in Windows due to memory corruption (which doesn't make me trust the aggsoft sniffing driver, but its interface was a lot nicer).

[gerph]

Oh gosh, I've just realised why the bit order is odd in the newer model... the endianness is probably reversed. If it was the other way around then instead of being 0bgggBBBBBRRRRRGGG it would be 0bRRRRRGGGgggBBBBB which makes a lot more sense, and would be marginally easier to construct. This is what I get for trying to decode things at 3am ! :-D

[mathoudebine]

Hi @gerph!
By decompiling the vendor software for rev. B / flagship displays, i found the meaning of the characters in the Hello command answer:
Vendor code:

                if (buffer2[0] == (byte) 202)
                {
                  if (buffer2[1] == (byte) 72)
                  {
                    if (buffer2[2] == (byte) 69)
                    {
                      if (buffer2[3] == (byte) 76)
                      {
                        if (buffer2[4] == (byte) 76)
                        {
                          if (buffer2[5] == (byte) 79)
                          {
                            if (buffer2[8] == (byte) 0)
                            {
                              if (buffer2[9] == (byte) 202)
                              {
                                this.comName = list[index2];
                                Settings.Default.comName = this.comName;
                                if (buffer2[6] == (byte) 10 && buffer2[7] == (byte) 1)
                                  Settings.Default.version = "A01";
                                else if (buffer2[6] == (byte) 10 && buffer2[7] == (byte) 2)
                                  Settings.Default.version = "A02";
                                else if (buffer2[6] == (byte) 10 && buffer2[7] == (byte) 17)
                                  Settings.Default.version = "A11";
                                else if (buffer2[6] == (byte) 10 && buffer2[7] == (byte) 18)
                                  Settings.Default.version = "A12";
                                Settings.Default.Save();
                                flag = true;
                                break;
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }

As you can see in this C# code, buffer[6] and buffer[7] are used to define a "version"
buffer[8] is expected to be always 0

The 4 available versions define hardware capabilities like you suggested:

• A01: HW revision B - brightness 0/1
• A02: HW revision "flagship" - brightness 0/1
• A11: HW revision B - brightness 0-255
• A12: HW revision "flagship" - brightness 0-255

The 'flagship' version has RGB leds on the backplate, whereas version B does not.
The A01 and A02 do not have brightness range, they accept only 0/1 as the brightness level:
Vendor code:

      byte[] buffer = new byte[10]
      {
        (byte) 206,
        !open ? (byte) 1 : (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 0,
        (byte) 206
      };

0 means full brightness, 1 mean no brightness

I made the changes for this project on this commit if you're interested:
dee8833

[gerph]

Oh wow, that's amazing work ! I'll certainly integrate those into my version as well.

Strange that they have a version that has binary lights, but I guess it just gives them options.

[gerph]

I don't suppose your disassembly found anything about inverting the bitmaps horizontally and vertically in addition to the landscape/portrait setting?

I tried sending a whole load of values in different positions in the payload and with different bit values, but I couldn't work out how it was meant to work.

That said, I see that the orientation you've discovered for the model A was using 0,1 and 100, 101 - I never tried 100/101, because I'd have assumed they'd have used a bit flag rather than +100 to indicate flipping. I'll give that a while later and see if that works but I won't be surprised if it does not.

[mathoudebine]

Here's what i found about reverse orientations (reverse portrait and reverse landscape) :

The default basic display orientation (landscape or portrait) is defined based on the theme background picture (if the height is greater/less than the width...)

Then there is this combo box that allows to reverse this orientation:

When the reverse orientation (180°) is selected, all images are rotated before being sent to the display (command 0xCC = 204). So the reverse orientations are in fact software-managed.

Corresponding code (see "rotate" boolean):

    public void SendComData(
      int x,
      int y,
      int xPixelNum,
      int yPixelNum,
      int xAll,
      int yAll,
      Bitmap sendImg,
      bool rotate = false)
    {
      int num1 = x;
      int num2 = y;
      Bitmap bitmap = new Bitmap((Image) sendImg);
      if (rotate)
      {
        num1 = xAll - x - xPixelNum;
        num2 = yAll - y - yPixelNum;
      }
      try
      {
        byte[] buffer1 = new byte[10]
        {
          (byte) 204,
          (byte) Math.Floor((double) num1 / 256.0),
          (byte) (num1 % 256),
          (byte) Math.Floor((double) num2 / 256.0),
          (byte) (num2 % 256),
          (byte) Math.Floor((double) (num1 + xPixelNum - 1) / 256.0),
          (byte) ((num1 + xPixelNum - 1) % 256),
          (byte) Math.Floor((double) (num2 + yPixelNum - 1) / 256.0),
          (byte) ((num2 + yPixelNum - 1) % 256),
          (byte) 204
        };
        if (this.serialPort != null)
          this.serialPort.Write(buffer1, 0, 10);
        int count = xPixelNum * yPixelNum * 2;
        byte[] buffer2 = new byte[count];
        for (int x1 = 0; x1 < yPixelNum; ++x1)
        {
          int num3 = (yPixelNum - 1 - x1) * xPixelNum * 2;
          for (int y1 = 0; y1 < xPixelNum; ++y1)
          {
            Color pixel = bitmap.GetPixel(x1, y1);
            byte num4 = (byte) ((uint) (byte) ((uint) pixel.R & 248U) | (uint) (byte) ((uint) pixel.G >> 5));
            if (rotate)
              buffer2[num3 + y1 * 2] = num4;
            else
              buffer2[count - (num3 + y1 * 2) - 2] = num4;
            byte num5 = (byte) ((uint) (byte) (((int) pixel.G & 252) << 3) | (uint) (byte) ((uint) pixel.B >> 3));
            if (rotate)
              buffer2[num3 + y1 * 2 + 1] = num5;
            else
              buffer2[count - (num3 + y1 * 2) - 1] = num5;
          }
        }
        if (this.serialPort == null)
          return;
        this.serialPort.Write(buffer2, 0, count);
      }
      catch (Exception ex)
      {
        Console.WriteLine(ex.Message);
        Console.WriteLine("Com Image Exception");
        this.comDataQueue.Clear();
        this.reset();
      }
      finally
      {
        bitmap.Dispose();
      }
    }

example of call for this function to set the background picture at start:

this.comData.SendComData(0, 0, this.backgroundImg.Height, this.backgroundImg.Width, this.backgroundImg.Height, this.backgroundImg.Width, this.backgroundImgProcessed, this.screenRotate);

Since i had some time, i implemented the changes in the feature/system-monitor branch: eda5d3c