Developing widgets for chumby

From Chumby Wiki
Revision as of 12:03, 22 January 2007 by 24.30.157.165 (Talk)

Jump to: navigation, search

Summary

Widgets for the Chumby are developed for Adobe Flash Lite 2, possibly with Flash Lite 2.1 being used in the future. Flash Lite 2 has a feature-set like that of Flash 7, meaning no Flash video (as the device alone isn't fast enough to play back video at a usable rate anyhow, though future implementations may allow for the use of the Alpha Chumby's processor's hardware accelerator for video).

As of right now,

  • PNG and GIF image format aren't included. This could change in the future.
  • Progressive JPEG support isn't included.
  • It does not support video (Sorenson or On2) or Nelly Moser audio
  • It does not support LocalConnection or XMLSocket.
  • It does not support the Flash Communication Server
  • There are no device fonts - all fonts must be embedded in the movie
  • Playback of external audio files (MP3, etc) are not yet supported.

Also important for developers to note is that the device's current input system is a touchscreen, meaning that mouseMove events will only occur while mouseDown (equivalent on current computers of only being able to move the mouse while holding a mouse button), which may/will have some effect on how your programs operate.

For efficiency's sake, Flash Lite downsamples images and embedded fonts, so avoid resizing images and small serif fonts, as detail will be lost.

Developers should avoid "play" or "repeat" buttons, and whenever possible opt for as little complex interactions with Chumby as possible; it is recommended that widgets need no human interaction. Text entry should also be avoided whenever possible, as the Chumby's text input system is crude at the moment, and most methods of touch screen text input are covered by patents and thusly won't be included in Chumby. (Most on-screen keyboards need more space than Chumby has, or are designed to operate with a stylus, which is not recommended for use with Chumby).

Performance and optimizations

The protoype chumby has a 266Mhz ARM9 processor - similar in performance to a low-end Pentium desktop. There are certain tricks and techniques that can make the difference between a sluggish movie and one that works smoothly.

  • Change property values only when necessary. The very act of changing a property can result in a redraw of that object, even if the value itself hasn't changed. Be sure to check to see if changing the property is necessary before changing it - for instance, don't change the _rotation value of the hour hand of a clock if it hasn't actually moved.
  • Change dynamic text only when necessary. If the text isn't actually different, don't change it.
  • Reduce the size of images by playing with compression settings. Make the images as small as possible with acceptable quality.
  • Reduce the size of audio by playing with compression settings. As with images, try for small size with acceptable quality.
  • Simplify vector graphics. See if you can use Flash's curve simplification to reduce the number of points and curves. For static graphics, you might even be better off creating bitmaps and using them instead.
  • Avoid layered translucent areas. Piling on a lot of translucency is expensive. You may be better off using PNGs with these effects already composited. If you can use masks, use them.
  • Avoid gradients. Gradient fills, particularly radial gradients and gradients with translucency, are expensive. Use them sparingly.
  • Avoid full-frame animation. Animation that changes large areas of the display can slow down the animation - try to keep the percentage of the screen being updated as small as possible for any given frame.
  • Use tweening sparingly. Try to keep tweening to one or two small objects at a time. Don't use shape tweens unless absolutely necessary - they're expensive to compute.
  • Avoid processing a lot of data in a single frame. Try queuing data in an array and process one item per frame. It complicates things a little bit, but will result in smoother operation and will avoid Actionscript timeouts.
  • Avoid lots of text. Lots of small text can be expensive to render. The chumby is meant to be read from a few feet away, so you really should be using large text anyway.
  • Don't do too many network fetches at the same time - Flashlite limits the number of fetches intiated on the same frame, and the total number of fetches in progress at the same time. Considering queuing fetches and intiating new fetches when the previous ones complete.

Leaks

Even Flash can be subject to memory leaks - the most common way to create a memory leak is to allocate XML objects as local variables. For instance, the code

  function someFunction() {
    var x = new XML();
    x.onLoad = doSomething;
    x.load("http://some.url");
  }

...will result in a memory leak. XML objects should be assigned to movieclip properties, and assigned to undefined when no longer used.

Eventually, memory leaks will cause the chumby to crash, although not always in the offending widget.

Another class of leaks is "interval" leaks. The use of setInterval() without corresponding clearInterval() will result in interval objects accumulating. Since widgets can be unloaded at any time, it's possible that the opportunity to clear an interval may not happen. Therefore, intervals should be avoided within widgets. The typical symptom of an interval leak is the widget seeming to get too many interval callbacks, although eventually the CPU will get overloaded with too many interval firings, and the chumby becoming extremely sluggish.

Widget Environment

Widgets are given a set of parameters about the environment of the chumby: For example:

this['_chumby_movie_url']='http://www.chumby.com/xml/movies/12345678-1234-1234-1234-123456781234'
this['_chumby_chumby_id']='87654321-4321-4321-4321-432187654321'
this['_chumby_chumby_name']='TestChumby'
this['_chumby_profile_id']='ABCDEF12-ABCD-ABCD-ABCD-ABCDEF12345678'
this['_chumby_profile_name']='Default'
this['_chumby_user_id']='FEDCBA9876-FEDC-FEDC-FEDC-FEDCBA98765432'
this['_chumby_user_name']='johndoe'
this['_chumby_widget_instance_href']='http://www.chumby.com/xml/widgetinstances/12121212-1212-1212-1212-121212121212'
this['_chumby_widget_instance_id']='12121212-1212-1212-1212-121212121212'
this['_chumby_base_url']='http://www.chumby.com'

A widget that has a configuration dialog may create additional values - for instance, a horoscope widget might create:

this['sign'] = 'Scorpio'

Security

The widget has a security sandbox similar to a movie running in a browser plugin - an external source of content should have an appropriate crossdomain.xml file to expose content to the widget. For more information, please see the following Flash Player Technotes and Articles:

One should also build widgets so they can be loadMovie()'d into another "virtual chumby" movie at some level other than _root - this means you should try to avoid adding random properties to _root or the built-in objects.

In general, widgets should be under 100K in size in order to reduce download time and use the minimum of storage in the device itself.

Testing a widget locally

The chumby itself has no substantial local storage, so you should not expect to store widgets on the chumby. The protoype does allow write access to the built-in NAND Flash, however, there's isn't much free space and it will be entirely overwritten by firmware updates. Future versions of the chumby will be read-only. If you want to run Flash movies on the device itself without uploading to the chumby servers, you'll need to do it from a USB mass storage device plugged into the back of the chumby.

  • Get a USB dongle, formatted as VFAT, and put your Flash movie on the top level.
  • Create a script file on the dongle called "debugchumby" with the contents
#!/bin/sh
/usr/bin/chumbyflashplayer.x -i /mnt/usb/<name of flash file>.swf

The file must use UNIX-style line terminations, and the last line must be terminated.

  • Plug the dongle into the USB port on the back of your chumby
  • Power it up - after the opening animation, your widget should run

Uploading a widget

You can upload a widget to our service which you can then use on your chumby. You will need to have the SWF for your widget, and a 80x60 pixel JPEG thumbnail image which represents your widget.

  • Create an account with chumby.com, if you haven't already
  • Log in - you will probably be redirected to a page that offers to let you register a chumby, or a list of your current physical chumbys
  • Select "my widgets" from the "my chumby" menu
  • Click the "upload a widget" link
  • Fill in the information

Widgets should be under 100K in size in order to reduce download time and use the minimum of storage in the device itself.

A widget uploaded using this mechanism will show up in the widget mix configuration, and can be selected just like any other widget, however, these uploaded widgets will only be accessible from your account. In the future, you'll be able to share these widgets with your friends, and even submit them to chumby for public access.

Using "virtual chumby"

The website has a mechanism for testing widgets in a Flash-based simulation of the widget playback of a real chumby. You do not need a physical chumby to utilize this functionality.

  • Create an account with chumby.com, if you haven't already.
  • Log in - you will probably be redirected to a page that offers to let you register a chumby, or a list of your current physical chumbys.
  • Select "my widget mixes" from the "my chumby" menu.
  • Click the "virtual chumby for this widget mix" link for one of the widget mixes - you should have at least one called "Default".

You can add widgets to any given widget mix. Both the real and virtual chumbys will check in for updated widget mixes every five minutes or so, or you can simple restart the Flash movie by reloading the virtual chumby's page.

Example widgets

Here are some Flash source (.fla) files for various types of widgets:

Using SCP to Test Applications

Do you know the joke about the how to avoid getting lost and stranded on a forest trek? Bring some vermouth and gin with you, and if you get lost, sit on a tree stump and mix 3/4 cup gin with a 1/4 vermouth and from all around the forest, people will jump at you and say, "that's no way to mix a martini"... and you're saved!

In a similar vein, I'm putting up how I was able to get a edit/compile/debug cycle going for the Chumby in the hopes someone who can improve it, "jumps out".

I was able to develop and test a few FlashLite apps using the method of placing a file on the USB key and moving and back and forth to the Chumby. This got old quickly, so I found this (somewhat) better workflow. Although SSH is available using the method above, so you can remotely open a shell from your development workstation, scp is not on the chumby.

  • Create a VFAT formatted usb key as described above.
  • Create a executable file on the dongle called "debugchumby" with the contents
#!/bin/sh
/bin/sshd
  • Test that this is working by restarting the Chumby and ensure you can ssh into this as root from your development box. Since root doesn't have a password, it's a good idea to remove the key and reboot after you finish development.
 ssh -l root chumby

You should get a root prompt at this point. Note, I've got dnsmasq running so my chumby has a 'name', it might just be an IP address for you (e.g. ssh -l root 192.168.0.XXX)

  • Note where your usb key is mounted on the chumby - it's /mnt/usb for me (and likely for you)
  • Put usb key back in your development box.
  • Get a copy of scp either by compiling it for ARM-linux, or find a precompiled ARM version. For instance, for Zaurus, there an scp in the OpenSSH package: http://www.killefiz.de/zaurus/showdetail.php?app=1035. The files can be unzipped & untarred, eventually locating scp.
  • place scp in the usb key root directory.
  • place usb key back in the chumby, and then reboot the chumby again
  • cd to /usr/bin on the chumby and create a symlink to the scp on /mnt/usb/. You could also just copy scp to /usr/bin, and I'm sure there's some clever way to set the path to have the /mnt/usb/scp, but I didn't bother (can be done with PATH=$PATH:/mnt/usb)

(on Chumby)

 cd /usr/bin
 ln -s /mnt/usb/scp /usr/bin
 cd /mnt/usb
 chmod +x scp
  • from your development box terminal window, as sudo (or root), you should now be able to copy a file
 sudo scp <filename> chumby:
  • from your chumby terminal window, you should now see the file you've just uploaded in /mnt/usb/<filename>

Good luck! This is much easier than moving the usb key back and forth!

  • Notes: sftp_server can be done similarly. This would be nicer because there's some nice GUI front-ends for it. however, I'm not able to get it working with the few minutes I've spent on it. I believe it has something to do with the sextopus ascii logo that is creating a problem for the sftp initiation, according to some googling on sftp_server failure modes. I'd like to temporarily remove this ascii art and see if it helps (where is it being called from?). Or maybe not...
  • Clearly you can save a step above by putting the scp on the key first. I thought it was worth getting ssh running first...
  • Samba would be very nice, you could save the flash swf on the chumby right from Flash Professional.
  • File Access Monitor to detect newly dropped SWF file, and automatically reinvoke Chumbyplayer.x

-hbchumby

Iterating With chumbyflashplayer.x

Now that you can move files to the Chumby easily, how can you iterate? What I did is create a similar script on the Chumby (in /mnt/usb) that kills the current Chumby player and restarts it with my test application:

 #!/bin/sh
 rm /tmp/flashplayer_started
 killall chumbyflashplayer.x
 /usr/bin/chumbyflashplayer.x -i mytestapp.swf

The chumby has a watchdog timer that looks for lockups in the FlashLite Player or Control Panel - deleting /tmp/flashplayer_started prevents it from relaunching the player.

-hbchumby

Flash Access to Sensors

The Chumby Flash Player has some ASnative calls to access the various sensors

NOTE: These are very likely to change in future releases - use at your own peril.

Touchscreen

Normally, the touchscreen is calibrated by the user using the Control Panel, however, raw coordinates are available.

_rawX = ASnative(5,10); // get the last raw touchscreen X coordinate
_rawY = ASnative(5,11); // get the last raw touchscreen Y coordinate
trace('x:'+_rawX()+', y:'+_rawY());

Bend Sensor

The bend sensor is currently used by the Control Panel to bring up the controls.

_bend = ASnative(5,14); // get the "last" bend sensor reading
trace(_bend());
_bent = ASnative(5,25); // get the "bent" flag (0/1)
trace(_bent());
_bendAverage = ASnative(5,28); // get the 10-second running average of the bend sensor

Light Sensor

_light = ASnative(5,15); // get the ambient light sensor reading
trace(_light());

Display

This allows you turn off the display, perhaps for power reasons.

_getLCDMute = ASnative(5,19); // get the value of the LCD "mute"
_setLCDMute = ASnative(5,20); // set the value of the LCD "mute"
trace(_getLCDMute());
_setLCDMute(1);

The "brightness" values are used by the Linux kernel to automatically dim the display. There are 16 subranges, each with its own threshold and associated brightness values.

_getBrightnessForRange = ASnative(5,21); // get the display brightness for the indexed light level  range
_setBrightnessForRange = ASnative(5,22); // set the display brightness for the indexed light level range
for (i=0;i<16;i++) trace(_getBrightnessForRange(i));
_setBrightnessForRange(5,255);
_getBrightnessThresholdForRange = ASnative(5,23); // get the threshold for the indexed light level range
_setBrightnessThresholdForRange = ASnative(5,24); // set the threshold for the indexed light level range
trace(_getBrightnessThresholdForRange(5));
_setBrightnessThresholdForRange(5,24);

Speaker

This cuts off audio to the built in speakers, but still allows audio through the headphone jack.

_getSpeakerMute = ASnative(5,17);
_setSpeakerMute = ASnative(5,18);
trace(_getSpeakerMute());
_setSpeakerMute(1);

DC Power

This tells you whether the device is connected to the DC power adaptor, or if it's running on the 9V battery. Currently, the Control Panel will turn off the display if the device is running on battery.

_dcPower = ASnative(5,16);
trace(_dcPower());

Most Common Problems

The most common problems with widgets that seem to fail on the chumby are:

  • The movie is the wrong version - FlashLite 2 will not play Flash 8 or Flash 9 movies.
  • The movie does not have sufficient security privileges - crossdomain files may not exist on the external content servers
  • The movie is not written to be loaded into another movie - this is often the case with movies with preloaders that reference _root or _level0 - if you need something globally accessible, consider making it a property of Object.
  • The movie uses device fonts - be sure all the required fonts are embedded
  • The movie has the wrong frame rate - the chumby wants widgets to be 12fps
  • The movie has the wrong dimensions - the widgets should be 320x240
  • The movie contains Flash Video - FlashLite 2 does not support Flash Video