Project Logs

Collapse

• Controlling speakers with AHK SoundGet, SoundSet

  WJCarpenter • 04/16/2021 at 23:35 • 0 comments

  AutoHotKey includes some built-in commands for controlling audio devices on a PC. Although there are several device and control types, I could only get it to work with "master" speakers. There's a lot I don't understand about how Windows thinks about audio devices, but using "master" for the speakers is OK with me.

  • Most applications provide no control for audio output, and most also don't even provide any UI feedback on the state of audio output. They seem to feel like it's Somebody Else's Problem.
  • I use a variety of different "speakers" from time to time (multiple headphones, built-in laptop speakers, HDMI audio to my monitor). I'm skeptical about being able to control the right device at the right time. (In fact, even as a PC user operating the controls with my mouse, I find things get kind of bonkers sometimes.)
  • The "master" control provides visual feedback in the little speaker icon in the system tray.

  The AHK SoundSet command will let you explicitly turn speakers on or off, and it also will let you toggle things. Even though I have disparaged hotkey toggles, I found it convenient to use it for implementing speaker control in this AHK function. SoundGet returns either "On" or "Off" for the state of muting of the speakers. To unmute the speakers, we want to "turn off the mute function".

  ; Control-Shift-F11 mute speakers
  ^+F11::
    SpeakerControl("On")
  return
  ;;;;;;;;;;;;;;;;;;;
  ; Control-Shift-F12 unmute speakers
  ^+F12::
    SpeakerControl("Off")
  return
  
  SpeakerControl(StateWanted)
  {
    ; mute state "On" means the speakers are muted
    ; mute state "Off" means the speakers are not muted
    SoundGet, MUTE_STATE ,,MUTE
    if (StateWanted != MUTE_STATE)
    {
      ; this toggles mute state, so we only do it if appropriate
      SoundSet, +1, ,MUTE
    }
  }

• These keys are so hot

  WJCarpenter • 04/16/2021 at 23:05 • 0 comments

  I originally thought that the computer side of this was going to be the easy part. And, I guess it is or it isn't, depending on how you look at it.

  If you are only interested in using a single video conferencing application, and if you can live with the ambiguity of keyboard shortcuts that act as toggles, then you can customize the RemuterMCS keyboard codes on the remote device to do your bidding. For example, here are the keyboard shortcuts I found for some popular video conferencing applications:

  ; Google Meet default Windows shortcuts
  ; https://support.google.com/a/users/answer/9896256
  ; microphone toggle: Control-d
  ; camera toggle:     Control-e
  
  ; Webex Meetings default Windows shortcuts
  ; https://help.webex.com/en-us/84har3/Cisco-Webex-Meetings-and-Cisco-Webex-Events-Accessibility-Features
  ; microphone toggle: Control-m
  ; camera toggle:     Control-Shift-v
  
  ; Zoom default Windows shortcuts
  ; https://support.zoom.us/hc/en-us/articles/205683899-Hot-Keys-and-Keyboard-for-Zoom
  ; microphone toggle: Alt-a
  ; camera toggle:     Alt-v
  
  ; Microsoft Teams meetings and calls
  ; https://support.microsoft.com/en-us/office/keyboard-shortcuts-for-microsoft-teams-2e8e2a70-e8d8-4a19-949b-4c36dd5292d2
  ; microphone toggle: Ctrl+Shift+M
  ; camera toggle:     Ctrl+Shift+O
  

   I really don't care for the simple toggles that the standard hotkeys use, and I myself use multiple conferencing applications. So, I started looking at my original plan, which was to have AutoHotKey do the heavy lifting. 

  That's still my plan, but it takes a little detective work to figure out the details. I was hoping to have a recipe so anybody could figure out what they needed to paste into an AutoHotKey script, but it turns out that there are a lot of variables. Part of the problem, which I didn't anticipate, is that AHK's ability to find things in controls inside an application probably depends on the use of technology that isn't in common use any more. For example, looking at "all text in a window" pretty much comes up with nothing useful. I had originally planned to look for text "Mute" or "Unmute", for example, to know the state of the microphone mute.

  In the next few project logs, I'll describe some of the techniques I am using to find and change states of devices. I don't yet have a simple recipe, but maybe that will arise as I document my trail. Ironically, the least useful of the 3 devices, the speaker, is the simplest to control via AHK.

• Brief video demo of RemuterMCS

  WJCarpenter • 04/11/2021 at 19:06 • 0 comments

  The user interface is done enough, and I finally got around to making a short demo video of the gadget in action.

  It's just the basics of operating it, so I don't go into detail about all the configuration options and so on.

• Is this radio on?

  WJCarpenter • 03/29/2021 at 02:18 • 0 comments

  I've pretty much finished fooling around with the firmware on the M5StickC. It's been great fun clicking the buttons and seeing the AHK pop-ups on my computer screen, but I now want to work on making AHK actually do the things it says in the pop-ups.

  I got a M5StickC Plus, which has a larger display, and adapted both of the screen renderings to the larger size. The code now runs fine on either kind of device, but you do have to say which it is in a RemoterMCSconfig.h before building and uploading the firmware. They use different TFT video controller chips. I don't think it's technically possible to have one firmware that serves both devices. If I ever figure out a way to do that, I can eliminate the configuration step.

  Here is a photo comparing the two devices, M5StickC Plus on the left, M5StickC on the right. (I don't know if they are intentionally different shades of orange or if it's just variation from different manufacturing runs.) For both displays, the dashed vertical bar on the right side is the battery indicator. The stripe on the left side is Received Signal Strength Indicator (RSSI) of the connected computer. The RSSI varies quite a bit, but it does give a coarse indication of strong or weak signal. (I don't understand why, but it seems a Bluetooth server, which is what these are, must ask the Bluetooth client for its RSSI value. Since both ends are receiving signals, I expected to be able to ask my radio what RSSI it had seen. But, apparently not.)

  One thing I have to come back to and hope to improve is battery life. There's a lot I don't know about Bluetooth LE, even though I know a lot more today than I did when I started this project. I want to figure out how to put the ESP32 into light or deep sleep most of the time to save power. I know how to do that, but it kills my Bluetooth conversation with the computer. This could be something that needs a code change in the ESP32-BLE-Keyboard library, or it may be something I can overcome with other BLE or ESP32 calls in conjunction with that library. I have to study that some more.
  

  My simple battery longevity testing was disappointing, but fairly conclusive. Regardless of how much delay() I put into each iteration of the Arduino loop(), the lifetime of the M5StickC battery is between 40 and 50 minutes. It's not too surprising that the amount of delay() doesn't matter too much since the processor continues to run at full speed. On the ESP32, delay() just makes an RTOS call to suspend the foreground task for that amount of time. The clock keeps ticking.

  I repeated the longevity timing with a modified program that didn't do anything with the Bluetooth radio. It wasn't even initialized. I got between 110 and 120 minutes that way. More than double, less than triple. It gives me some encouragement that if I can crack the light/deep sleep puzzle so I can continue BLE when I wake up, I should be able to get a lot of battery mileage. When starting from scratch, it can take from 1-10 seconds to establish the Bluetooth connection, which is pretty unacceptable for this use case. 

  I just need to figure out how the Bluetooth remote control folks do it. I'm sure their devices sleep most of the time. I paged through the BLE documentation intended to facilitate the low power part, but there's a lot of jargon and a lot of layers of APIs, so it's taking me a while to match things up. I haven't yet come across a good example, although there are plenty of examples that don't worry about power. 

  EDIT: I figured out how the remote control people do it. They use more special purpose Bluetooth chips that are optimized for it. The ESP32 will probably never be able to be that efficient, but it should be able to do a lot better than it does for me right now.

• Battery percentage, note 3

  WJCarpenter • 03/26/2021 at 04:01 • 0 comments

  I suppose it's due to the relatively small battery, but things seem to go pretty unevenly as it charges and discharges. I am still using a strictly linear approximation for how much battery remains, but I'm a little skeptical that it's correct or any more than grossly meaningful. I draw a bar along one of the long edges of the display to show the remaining battery.

  I verified by checking registers in the AXP192 that the automatic shutdown voltage is 3.0v, as documented for the M5StickC. FWIW, the L1 and L2 warning levels are 3.45v and 3.40v, respectively, but they are not used for anything by default.

  With my previously described technique of recording the high and low voltages observed, the lowest I ever saw was a little above 3.4v. I suspected it was the granularity of the 10 minute intervals that was givng me that puzzling value, so I changed the timing to observe and record the low voltage every 40 seconds, and that showed me a low voltage of 3.0778v. That pretty much confirms that the AXP192 is behaving as expected and shutting things down at 3.0v.

• Bright breath

  WJCarpenter • 03/26/2021 at 03:53 • 0 comments

  For the M5StickC, the brightness of the TFT display is not controlled the way you might expect, with a PWM operation on a GPIO pin from the ESP32 processor. Instead, the backlight is wired to an output pin of the AXP192 power controller. The voltage supplied by that pin can be programmed to provide 1.8v to 3.3v in 0.1v increments.

  M5's Arduino IDE library for the M5StickC provides an AXP192 method called ScreenBreath(). I don't know why it's called that, but what is does is set a value to control the output voltage from that pin on the AXP192. That saves you a small amount of trouble in doing it yourself over the I2C interface. Interestingly, even though the AXP192 would accept values 0 - 15, the M5 API limits the upper range to 12. (I suspect that's based on how much voltage the TFT backlight LED would tolerate. The M5StickC product page says "2.8v max200ma". The AXP192 default output voltage is 2.8v. Maybe not worth sacrificing a device to find out what happens when you overdrive the backlight LED. :-) )

  I did some experiments, and the display is completely dark with any value of 7 or below (at least for my eyes with the M5StickC that I'm testing).

  So, realistically, the usable values are 8, 9, 10, 11, 12. 

  I provide a configuration item for setting the display brightness in my sketch. I don't limit it at either end, but the above limits are still there.

• Battery percentage, note 2

  WJCarpenter • 03/22/2021 at 02:29 • 0 comments

  I know a little bit about the battery parameters in the M5StickC now. The power controller is the AXP192. The manufacturer makes a comprehensive AXP192 data sheet available. Unfortunately for me, most of it is in Chinese. With the help of Google translate and code published by M5, I was able to figure out quite a bit. (There is an English data sheet available, but it's only 4 pages, compared to more than 50 pages for the Chinese data sheet.)

  • The AXP192 targets charging to a battery voltage of 4.20v, plus or minus 5%.
  • M5 has configured the AXP192 to automatically shut things down at 3.00v.
  • M5 provides an API called GetBatteryStatus(), but it returns an 8-bit value that is a collection of flags from an AXP192 register. The datasheet told me what each bit was.

  (M5's AXP192 class includes helpful methods for reading and writing AXP192 register values. Alas, they are all private methods, so I can't use them directly to get things that M5 didn't expose. It's not insurmountable, but it's inconvenient.)

  I found out experimentally that if I plug in the device when it's a little below 4.20v but within 5%, the AXP192 doesn't start charging it. And, since the AXP192 switches things to run completely on the USB current, the battery won't discharge and things will stay that way. OTOH, when the battery is being charged, it only got up to about 4.18v. That's within that 5% range (in fact, only about a half percent difference), but I expected it to be more on target. I don't know if that's due to AXP192 logic or if my specific battery won't take more charge.

  I recorded the voltage as the battery discharged, as described in my previous note. The lowest recorded voltage was 3.53v. That's quite a bit higher than the 3.00v shutdown voltage I expected. I'm going to run a few more charge/discharge cycles to see if that low number changes. When the numbers settle down, I will probably still use a linear calculation for the percentage. That's probably close enough for this kind of experience. What you really want to know is when the device is going to quit due to a low battery.

  The application logic I am using to record the voltage range at this point:

  • For any new lower voltage, record it regardless of charging or plugged-in status.
  • If the device is plugged, not charging, and the difference from the measured voltage is more than 5% different from 4.2v, record it as a new high water mark. (I'm not really sure what happens if the AXP192 can't get the battery up to the target voltage range.)

• Icons for everyone

  WJCarpenter • 03/20/2021 at 02:07 • 0 comments

  I picked out the UI icons for microphone, camera, and speakers. There are zillions of icons to look through at https://thenounproject.com/, all of which are better than I could make myself, even for this fairly simple shapes. I fiddled with different choices quite a bit by viewing them on the M5StickC display. To make it easy to try things, I wrote a simple sketch IconsAtPlay that is a bit too long to show here. It displays the icons in multiple ways, in both 80x80 and 40x40 sizes.

  To make it simple to get the sizes I needed in uniform square backgrounds, I started with SVGs and converted them to XBMs with everybody's favorite graphics manipulation tool, ImageMagick.

  I was originally going to overlay a "?" for the unknown status and a circle-slash for disabled status, but I didn't like the look of it. Instead, I just draw a diagonal line a few pixels wide for disabled status. I also draw a 1-pixel box around the icon when that target has focus on the user interface. The target with focus uses the 80x80 icon size, and the other targets use the 40x40 size.

  Here is a photo of the device (mocked up with the IconsAtPlay sketch) showing microphone muted, camera enabled, and speakers in an unknown state. The microphone target has focus.

  Below are the original forms of the icon graphics. All of the icons are licensed under Creative Commons 3.0. They're displayed here in black on a transparent background. I provide the colors when I render them on the device. 

  Microphone: Microphone by Chunk Icons from the Noun Project
  https://thenounproject.com/icon/829069/

  Camera:     webcam by Kawalan Icon from the Noun Project
  

  https://thenounproject.com/icon/2131644/

  Speaker:    Speaker by Alex Arseneau from the Noun Project
  

  https://thenounproject.com/icon/872876/

• Splash!

  WJCarpenter • 03/18/2021 at 15:57 • 0 comments

  I've designed a splash screen, which is the first thing you see when you power the device up. It's also the screen which tells you whether or not it's connected to Bluetooth, which is nice to know.

  (Instead of this sloppy picture, I had intended to read the pixel values from the display and show it here in screenshot fashion. I never got that to work, despite an encouraging comment in the library code. Reading pixels always came back as 0 values no matter what I tried.)

  I originally started working with the "built in" fonts from the M5StickC Arduino library because I didn't feel like messing around with figuring out how the fonts worked in the Adafruit GFX library, and I also wasn't sure it was worth the flash space. It turns out that the M5StickC "M5Display" already includes all that stuff behind the scenes, including all the standard Adafruit converted "FreeFonts". The space is already being consumed unless you go to some trouble to chop those fonts out. So, I switched over to using them and experimented until I got sizes and styles that I liked.

  What's the splash screen displaying?

  The middle line is the hardware Bluetooth address of the device, which is commonly called the MAC. When the device advertises itself for pairing, the part in blue is included in the device name. That's in case there are multiple devices floating around and you need to be able to tell them apart.

  The "connecting ..." part of "BT connecting ..." repeatedly types itself as an animation while connecting is taking place. Once connected, the entire line changes to a blue "BT connected". The display rests there for 1-2 seconds and then switches to the normal view showing the icons for the microphone, camera, and speakers.

  You might not notice that this display has a bias for people who hold the device in the right hand with the "M5" button away from them. There is a configuration option to rotate the text 180 degrees for those who prefer it that way. There is no performance difference either way.

  After I took the picture, I changed the middle line to use a sans serif font, which made it more readable. That's probably it for the visual design of the splash screen, except for overlaying the battery and signal strength indicators that will also be on the main screen. I might also move things around to make them more centered, but to be honest I didn't notice it being off-center so much until I saw this still photo. I'll make a video of the splash screen in action later. I'm pretty pleased with how it turned out.

• Battery percentage, note 1

  WJCarpenter • 03/17/2021 at 03:27 • 0 comments

  I plan to give some kind of visual indication of the amount of battery power remaining, but the problem is determining some kind of quantification for that. M5 does not provide a direct API to answer the question "how much battery left?", but they do provide APIs for the AXP192 power controller to get all sorts of interesting information.

  I naively thought there would be some kind of common knowledge way of taking LiPo battery voltage and translating that directly into what I want. And, it's possible that there is, but I didn't stumble across it. I did find quite a bit of discussion about it. Here is a link to one discussion of it. This person provided a handy table of voltages and percentages, but I'm kind of skeptical, because

  • In the first place, my fully charged battery voltage was a bit lower than their 100% figure, 
  • and I have no idea what my low-voltage is when the device shuts itself down. 
  • I'm also pretty sure the discharge rate isn't linear, but I don't know what it is instead.
  • As my device ages, the battery capacity will degrade.
  • And, finally, there could be variation so that your battery is different from my battery (or two of my batteries might be different from each other).

  Here is my tentative plan. I haven't implemented it yet, so it's subject to change.

  • I'll use the "Preferences" API to store some battery-related values in flash. 
  • When the device is fully charges (as detected from the power in and out values), I'll record the battery voltage if it's different by more than a few percent from the last "high" value I stored, whether higher or lower. I don't expect any higher "high" values, except for a little jitter, but I expect lower "high" values as the battery ages. 
  • Periodically, maybe every 10 minutes or so, I'll store the current battery voltage if it's lower by more than a few percent from the last "low" value I stored. 
  • I'll run the device on battery power until it shuts off, fully charge it, and I'll repeat that a few times.
  • Using those "high" and "low" values, I'll ignore science and assume a linear discharge between them and convert that to a percentage.

  Note to self: consult the wisdom of crowds about how LiPo voltages change as they degrade over time.

View all 18 project logs