Implementing a Remote Instrument Front Panel

How to make a clickable pixel-perfect remote web interface to control your embedded GUI device

Implementing a "Remote Front Panel" Using the Webserver

Some Mosaic controllers offer a touchscreen-based "Graphical User Interface" (GUI) that enables a user to control the instrument by touching buttons and menus on the screen. The software that runs the GUI is called the "GUI Toolkit" and is available as a pre-coded "kernel extension" library from Mosaic Industries. These GUI-based controllers include the QVGA Controller and the PDQScreen; the feature is not recommended for use on the QScreen due to hardware limitations that slow its ability to serve the screen image over the web.

A web-based "Remote Front Panel" can be added to a touchscreen-based controller using an EtherSmart or WiFi Wildcard based on the Lantronix XPort or WiPort web devices. The remote front panel enables instrument access management, allowing a user to control the instrumentation via a web browser. The browser displays a bitmapped image of the GUI screen exactly as it appears on the controller’s LCD (Liquid Crystal Display) as illustrated in Figure 1-14. Clicking on the web-based image with a mouse has the same effect as touching a button on the touchscreen: both the local screen image and the web image are updated. There is a delay before the new image appears on the web, because it takes time to extract, format and serve out the bitmapped image. Use of the remote front panel feature can expand the capabilities of an instrument, offering the possibility of web-based remote control, reporting, and diagnostics via an Ethernet or WiFi connection.

To implement this feature in your application, you can incorporate the source code presented in Appendix C (for C programmers) or E (for Forth programmers). The GUI Toolkit functions that enable this feature are described in a special section at the end of the EtherSmart/WiFi Wildcard Glossary document.

The remainder of this section describes how the remote front panel works.

Figure 1-14 Web-based remote front panel running on the QVGA Controller.

The HTML text shown in Listing 1-16 creates the image shown in Figure 1-14:

Listing 1-16 Remote front panel HTML source from gui_response_text.html.

<html><head><title>EtherSmart/GUI Remote Front Panel</title></head>
<body>
<H3>EtherSmart/GUI Remote Front Panel</H3><p>
<H4>Click on the screen image to operate the touchscreen remotely</H3><p><p>
<a href=“/gui_response.cgi”><img src=“/screen_image.bmp” ismap></a>
</body></html>

This file is made into a resource by the Image Converter program as described in the section above titled "Introduction to the Dynamic Webserver: Using the Image Converter to Create Web Resources", and in Appendix A. Briefly, we place the gui_response_text.html file along with the GUI demo program’s button graphics in a directory, invoke the Image Converter from the development environment, select the correct platform, check both "GUI images" and "Web Files", and click "Convert Files Now". The HTML in Listing 1 16 can now be referred to by the generated constants GUI_RESPONSE_TEXT_HTML_XADDR and GUI_RESPONSE_TEXT_HTML_SIZE; the constant names are based on the specified file name.

A standard set of <html>, <head>, and <title> tags start the web page generation in Listing 1 16. The <H3> header style is used to display the "EtherSmart/GUI Remote Front Panel" announcement. The <H4> tag formats the instruction to "Click on the screen image to operate the touchscreen remotely". The next line is the key to the technique:

<a href="/gui_response.cgi"><img src="/screen_image.bmp" ismap></a>

The ismap keyword at the end of the tag tells the browser that the image should be treated as an "imagemap". Any mouse click in the image causes the browser to send the x and y coordinates of the mouse click (relative to the upper left corner of the image) as a specially formatted query field appended to the URL. The EtherSmart/WiFi driver’s pre-coded HTTP_Imagemap function extracts the coordinates from the URL for use by the GUI Toolkit. The anchor tag declares an href reference to the URL /gui_response.cgi that is invoked each time the image is clicked. The handler for this URL invokes the HTTP_Imagemap function to extract the x and y mouse click coordinates, and passes them to the GUI Toolkit’s Simulated_Touch_To_Image function. This processes the coordinates and updates the screen as if an actual touch to the front panel had occurred. The <img> tag specifies that the image itself is served out by the /screen_image.bmp URL as specified by the image src keyword. The </body> and </html> tags complete the page source.

To start the web display, we need a starting web handler function that serves the initial screen image when the webserver is first connected to the controller. This web handler is called Remote_Panel_Start in Listing 1 17 and is designated as the home page invoked by either the / or /index.html URL. This function creates an HTTP header in the HTTP_Outbuf using the HTTP_Put_Headers function. It then calls the Init_Screen_Image function, which invokes the GUI Toolkit function Screen_To_Image to place the bitmap image in a buffer.

Because this web handler calls a GUI Toolkit function (Screen_To_Image), task synchrony issues come into play. The GUI Toolkit often runs in its own task, and the webserver runs in the Ethernet task. To avoid multitasking errors and resource conflicts, functions that involve both GUI and HTTP functions must be synchronized with one another. This is done automatically if some simple programming rules are followed:

Programming rules for the Ethernet/GUI Remote Front Panel on the EtherSmart/WiFi Wildcard

Each web handler function that invokes a GUI Toolkit function (Graphic_To_Image, Screen_To_Image, Simulated_Touch_To_Image, or Screen_Has_Changed) must be posted using HTTP_Add_GUI_Handler. The standard non-GUI function HTTP_Add_Handler must not be used for these handler functions.
Each web handler function that invokes a GUI Toolkit function must use one and only one call to HTTP_GUI_Send_2Buffers to send out the HTTP header and content.
Each GUI handler function that changes the screen appearance must call the Screen_Has_Changed function.
For QScreen Controllers only, the initialization statement Globalize_TVars(TVARS); must be invoked when the application starts up.

Following these rules allows the various tasks to work together in harmony to implement the remote front panel.

Let’s continue examining how the functions in Listing 1-17 work. We now have the initial screen image as shown in Figure 1-14 in the web browser window as placed by the Remote_Panel_Start handler function. Now the user clicks on one of the keypad buttons (say, the "7" button) in the image. This causes the following URL and formatted "imagemap" query string to be sent to the EtherSmart/WiFi Wildcard:

/gui_response.cgi?43,69

This URL is handled by the GUI_Response handler function whose source is shown in Listing 1-17. This handler calls the HTTP_Imagemap function which extracts the x and y values from the URL (43 and 69, respectively in this example). Because C functions can return only one parameter, the two 16-bit coordinates are combined into a 32-bit value that is declared as a TWO_INTS union as defined in the types.h file in the mosaic include directory of the C distribution. The handler stores x and y into separate variables, and passes them to the GUI Toolkit function Simulated_Touch_To_Image. This function treats the coordinates as those of an actual touch on the front panel, calls any GUI handlers associated with the button, updates the local display, and stores a bitmap image of the display into the specified buffer at BITMAP_BUFFER. GUI_Response then invokes HTTP_GUI_Send_2Buffers to serve out the HTTP header from HTTP_Outbuf, and the HTTP content from the resource string specified by GUI_RESPONSE_TEXT_HTML_XADDR and GUI_RESPONSE_TEXT_HTML_SIZE.

The web content served out by the GUI_Response handler function is shown in Listing 1-16. It uses the img tag to refer to the URL /screen_image.bmp. This URL is serviced by the Screen_Image_Response handler in Listing 1-17. When this handler is called, the bitmap image of the screen has already been created and is sitting in the BITMAP_BUFFER. The first 4 bytes at BITMAP_BUFFER contain a 32-bit count, with the data following. After generating the HTTP header using the standard function calls, the Screen_Image_Response function gets the size of the bitmap by invoking

image_size = FetchInt(BITMAP_BUFFER+2)

The remote front panel feature is limited to image sizes that can be represented as a 16-bit number, so the 2 most significant bytes of the count can be skipped. Two calls to HTTP_Send_Buffer serve out the HTTP header from HTTP_Outbuf, and the bitmap image data starting at BITMAP_BUFFER+4. Note that the Screen_Image_Response handler does not call any GUI Toolkit routines, so it need not follow the first two rules for Ethernet/GUI functions (shown in the box, above). Thus it can use two calls to HTTP_Send_Buffer instead of HTTP_GUI_Send_2Buffers, and it is posted via the HTTP_Add_Handler function in Install_GUI_Web_Handlers as shown in Listing 1-17.

Some additional steps are required to complete the remote front panel. These are additions to the GUI demo program that creates and manages the touchscreen-based image.

Place a call to the Ether_Check_GUI function in the application task loop that runs the GUI toolkit. The GUI toolkit function is called Service_GUI_Events on the QVGA controller and Service_Touch on the QScreen. Search for the GUI Toolkit routine (or its equivalent on other controllers) in the GUI demo program, and insert the following function call in the main task loop:
```
Ether_Check_GUI( E_MODULENUM ); // service remote front panel handler
```
Add a call to the Screen_Has_Changed in each GUI handler function that changes the screen appearance.

Please read the "Notes" at the bottom of Listing 1-17 for a reminder of these and other required steps.

The GUI demo program which implements the touchscreen display screen shown in Figure 1-14 is started by the GUI_Demo function. To start both the GUI demo program and the remote front panel feature, we invoke the high level function named Ether_GUI_Web_Demo (for an EtherSmart Wildcard) or WiFi_GUI_Web_Demo (for a WiFi Wildcard) in Listing 1-17. The definition of main can be edited to invoke the function that corresponds to your Wildcard (EtherSmart or WiFi). It calls Ether_Task_Setup_Default, Install_GUI_Web_Handlers, and GUI_Demo to initialize the required tasks and services. After calling this high-level initialization function, browsing into the EtherSmart/WiFi Wildcard’s home page (port 80 at the Wildcard’s IP address) displays the remote front panel in your browser.

For C programmers, the C_Ether_gui_demo_includer.txt file in the EtherSmart_WC_Code\Ether_GUI_Demo directory loads the install.txt and image_data.txt files and the C compiler output file for the Ethernet GUI demo program. For Forth programmers, the Ether_gui_demo_includer.4th file in the \Ether_Demo directory loads these files and the source code file for the Ethernet/WiFi demo program. These commented files provide a useful template to manage the loading of this demonstration program.

Listing 1-17 Excerpts from Ethernet/WiFi/GUI remote front panel source code.

// ********************* URL STRINGS **************************
char* slash_url_str = “/”; // synonym for home page url
char* index_url_str = “/index.html”; // home page url
char* screen_image_str = “/screen_image.bmp”; // screen image url
char* gui_response_str = “/gui_response.cgi”; // gui response url
 
// *********************** WEBSERVER ***************************
// Notes: this works very nicely with Opera, the recommended browser.
 
// the following is created using the Image Converter program, creates 2 constants:
// GUI_RESPONSE_TEXT_HTML_XADDR
// GUI_RESPONSE_TEXT_HTML_SIZE
 
void Screen_Image_Response( int modulenum )
// handler for /screen_image.bmp url. serves the screen image.
// assumes that Simulated_Touch_To_Image has run.
// xbuffer contains 32-bit count (NOTE!) followed by image data.
// marks as no-cache (always reloads).
// NOTE: this function is NOT a gui handler; it should be posted by http_add_handler.
{  xaddr http_outbuf_base = HTTP_Outbuf(modulenum);
   uint http_outbuf_size =  HTTP_Outbufsize(modulenum);
   uint image_size = FetchInt((xaddr) (BITMAP_BUFFER+2));   // get size of image
   uint http_header_size;
   HTTP_Put_Headers(http_outbuf_base, http_outbuf_size, TRUE, TRUE,
            HTTP_IMAGE_BITMAP_CONTENT);  // http header->outbuf
                  // params: xlbuf,maxbufsize,is_ok?,dynamic?,content_type
                  // dynamic bitmap image content ->outbuf;header is done
   http_header_size = FetchInt(http_outbuf_base);  // get size of http header
   HTTP_Send_Buffer(http_outbuf_base+2, http_header_size, modulenum);
   HTTP_Send_Buffer((xaddr)(BITMAP_BUFFER+4), image_size, modulenum);
}                 // send http header and bitmap image
 
void GUI_Response( int modulenum )
// clickable ismap version of the screen image that runs gui toolkit simulated touch.
// url = /gui_response.cgi
// this routine is called via ether_check_gui which runs in the application task
// that runs Service_GUI_Events.  it is posted using http_gui_add_handler.
{  xaddr http_outbuf_base = HTTP_Outbuf(modulenum);
   uint http_outbuf_size =  HTTP_Outbufsize(modulenum);
   uint http_header_size, x, y;
   TWO_INTS x_and_y;
   x_and_y = HTTP_Imagemap( modulenum); // get mouseclick x,y coordinates
   x = x_and_y.twoNums.msInt; // get x from most significant 16bits
   y = x_and_y.twoNums.lsInt; // get y from least significant 16bits
   Simulated_Touch_To_Image(BITMAP_BUFFER,BITMAP_MAXBYTES,BITMAP_SCREEN_FORMAT,x,y);
   HTTP_Put_Headers(http_outbuf_base, http_outbuf_size, TRUE, FALSE,
          HTTP_TEXT_HTML_CONTENT);  // http header->outbuf
                  // params: xlbuf,maxbufsize,is_ok?,dynamic?,content_type
                  // dynamic bitmap image content ->outbuf;header is done
   http_header_size = FetchInt(http_outbuf_base);  // get size of http header
   HTTP_GUI_Send_2Buffers(http_outbuf_base+2, http_header_size,
      GUI_RESPONSE_TEXT_HTML_XADDR, GUI_RESPONSE_TEXT_HTML_SIZE, modulenum);
}                 // send http header and content
 
_Q int Init_Screen_Image( void )
// if return value=0, image has been created
{  return Screen_To_Image((xaddr) BITMAP_BUFFER, BITMAP_MAXBYTES,
            BITMAP_SCREEN_FORMAT);
}
 
 
void  Remote_Panel_Start( int modulenum )
// serves an initial clickable ismap version of the screen image.
// url = /index.html
// puts initial version of screen into bitmap_buffer.
// this routine is called via ether_check_gui which runs in the application task
// that runs Service_GUI_Events.  it is posted using http_gui_add_handler
{  xaddr http_outbuf_base = HTTP_Outbuf(modulenum);
   uint http_outbuf_size =  HTTP_Outbufsize(modulenum);
   int http_header_size;
   HTTP_Put_Headers(http_outbuf_base, http_outbuf_size, TRUE, FALSE,
           HTTP_TEXT_HTML_CONTENT);  // http header->outbuf
                  // params: xlbuf,maxbufsize,is_ok?,dynamic?,content_type
                  // static html type->outbuf, cnt in first 2bytes; header’s done
   Init_Screen_Image(); // ignore return value; image has been created
   http_header_size = FetchInt(http_outbuf_base);  // get size of http header
   HTTP_GUI_Send_2Buffers(http_outbuf_base+2, http_header_size,
            GUI_RESPONSE_TEXT_HTML_XADDR, GUI_RESPONSE_TEXT_HTML_SIZE, modulenum);
                  // params: xbuf1,cnt1,xbuf2,count2,modulenum
}                 // send header&content, ignore numbytes_sent
 
#ifdef      __FABIUS__
#pragma option init=.doubleword    // declare 32-bit function pointers in code area
#include </mosaic/gui_tk/to_large.h>
#endif   // __FABIUS__
 
xaddr (*screen_image_response_ptr)(void) = Screen_Image_Response;
xaddr (*gui_response_ptr)(void)          = GUI_Response;
xaddr (*remote_panel_start_ptr)(void)    = Remote_Panel_Start; // home page
 
#ifdef      __FABIUS__
#include </mosaic/gui_tk/fr_large.h>
#pragma option init=.init       // return the initialized variable area to RAM;
#endif   // __FABIUS__
 
int Install_GUI_Web_Handlers( int modulenum )
// call this after ETHER_Task_Setup_DEFAULT  point browser to raw ip or to
//  ip/index.html to see the home GUI web page = remote front panel.
// urls are case sensitive.
// any other url’s serve out: page not found.
// returns nonzero error if too many handlers were added
// (limited by AUTOSERVE_DEFAULT_ROWS passed to ether_init)
// Some of the gui web handlers in this example are handled by the application task.
{  int error = 0; // we’ll OR error results together and return final result
   error |= HTTP_Add_GUI_Handler(STRING_XADDR(slash_url_str), strlen(slash_url_str),
            remote_panel_start_ptr, modulenum);
   error |= HTTP_Add_GUI_Handler(STRING_XADDR(index_url_str), strlen(index_url_str),
            remote_panel_start_ptr, modulenum);
   error |= HTTP_Add_Handler(STRING_XADDR(screen_image_str), strlen(screen_image_str),
            screen_image_response_ptr, modulenum);
   error |= HTTP_Add_GUI_Handler(STRING_XADDR(gui_response_str),
            strlen(gui_response_str), gui_response_ptr, modulenum);
   InitElapsedTime();      // start at zero so home page reports correct elapsed time
   return(error);
}
 
_Q void Ether_GUI_Web_Demo( void )
{    Ether_Task_Setup_Default();                // init EtherSmart WC, start ethernet task
Install_GUI_Web_Handlers(E_MODULENUM);     // setup web; ignore error flag
GUI_Demo();     // start the gui demo on local touchscreen
}                   // NOTE: for QScreen, GUI_Demo MUST call Globalize_TVars(TVARS);
 
_Q void WiFi_GUI_Web_Demo( void )
{    WiFi_Task_Setup_Default();                // init WiFi WC, start ethernet task
Install_GUI_Web_Handlers(E_MODULENUM);     // setup web; ignore error flag
GUI_Demo();     // start the gui demo on local touchscreen
}                   // NOTE: for QScreen, GUI_Demo MUST call Globalize_TVars(TVARS);
 
void main( void )
{                        // comment in the correct version for your wildcard:
Ether_GUI_Web_Demo();
//    WiFi_GUI_Web_Demo();
}
 
// NOTES: place a call to Ether_Check_GUI(E_MODULENUM); in the application task
// that invokes the GUI Routine
// (for example, after the call to Service_GUI_Events on a QVGA Controller system).
// Add a call to Screen_Has_Changed();
//  in each GUI handler that changes screen appearance.
// QScreen Controllers require a call to Globalize_TVars(TVARS); in the init function.
// When building tasks, do NOT include the statement NEXT_TASK = TASKBASE;
//  because we want to allow multiple tasks to be built.
// Type WARM or COLD before re-running the program.

This page is about: Clickable Remote Web Interface, Remote Instrument Control, Touchscreen GUI, Access Embedded Web Server from Any Web Browser – An EtherSmart or WiFi Wildcard implements a web-based "Remote Front Panel" that allows a user to operate the unit via a web browser. The browser displays a bitmapped image of the GUI screen, and clicking on the web-based image with a mouse has the same effect as pressing a touchscreen button.