Developing a complete context-aware application (CaMP) - Tutorial 10

This tutorial continues the series started a few weeks back, with a comprehensive example: developing the Context-aware Media Player (CaMP). Here, we describe and apply a development methodology, which details the steps required to design and implement a context-aware application using the MUSIC middleware.

Development methodology

The following steps describe a structured method for designing and implementing context-aware applications using the MUSIC Context System. It is recommended that before you try to follow this methodology, you study the Tutorials on the MUSIC Context System here.

The methodology consists of the following 6 steps:

1. Identify the relevant context types; in case some context types are higher-level, identify other elementary context types to build on them
2. For each elementary context type identify a “context sensor” plug-in and for each high-level context type identify a “context reasoner”
3. For each context plug-in, reuse an existing implementation or proceed to construct a new one
4. Develop the functional aspects of your application
5. Register your application to the relevant context types and implement the code that adapts the application accordingly
6. Pack your application in JAR file as per OSGi's bundle packaging specification
   

Furthermore, step 3 is also analyzed into the following sub-steps:

1. Implement (or reuse) the corresponding context model
2. Define the MANIFEST and XML descriptor of the “context plug-in”
   2.1. Define the standard properties like you would do for an OSGi component
   2.2. Define the “provided” context types
   2.3. Define the “required” context types [optional step for context reasoners]


3. Extend the abstract context sensor or context reasoner
   
4. Implement the “activate” and “deactivate” methods to start and stop the generation of context events respectively
   
5. Implement the “contextChanged” method to handle received context events [optional step for context reasoners]
   
6. Pack everything in a JAR file, as per the OSGi’s bundle packaging specification
   

Applying the development methodology to the Context-aware Media Player (CaMP)

The Context-aware Media Player is a simple application, which is defined as follows:

"CaMP is a simple media player, capable of playing audio or video media tracks, while exhibiting the following context-aware behavior:

• When the user is detected as entering the room (where CaMP is deployed), then resume playback
• When the user is detected as exiting the room, then pause playback"

We are going to develop an application that exhibits this kind of context-aware behavior using the proposed development methodology.

Step 1: Identify the relevant context types; in case some context types are higher level, identify other elementary context types to build on them

The first step is to analyze the relevant context types. In this case, the task it is easy because the application's verbal description is quite straight-forward. The main context type we are interested in is "whether the user is in the room". Thus we define the User-in-the-Room context type. This is a rather high-level context type, so we need to further refine it and decide how we can derive this context type.

While multiple approaches are possible, assume the following one: a user is assumed to have entered the room when his Bluetooth smart-phone is detected while at the same time some motion is detected by the web-camera on the deployment computer. Thus, we define two additional, elementary context types: Bluetooth devices and Motion. These two are combined to infer whether the user has entered (or exited) the room where CaMP is deployed.

The hierarchy of the context types is illustarted in the following figure:



Step 2: For each elementary context type identify a “context sensor” plug-in and for each high-level context type identify a “context reasoner”

The second step is to define a context plug-in for each context type. This comes naturally from the hierarchy of the context types, as illustrated in the figure above.

To detect the attached Bluetooth devices, we define a Bluetooth Sensor plug-in.
To detect motion in the room, we define the Motion Sensor plug-in
Finally, to detect whether the user has entered or exited the room, we define a context reasoner that takes input from the other two plug-ins; the User-in-the-Room Sensor plug-in

Notably, while the first two sensors have no context input (other than what is sensed from hardware, like the Bluetooth adaptor or the web-camera), the third one depends on input context types only. Thus, the first two can be constructed as context sensors and the third one as context reasoner.

Step 3: For each context plug-in, reuse an existing implementation or proceed to construct a new one

The next step is to reuse or develop a new plug-in for each one identified. As the construction of context sensors has already been covered extensively in Tutorial 8, we will here cover just the development of the User-in-the-Room context reasoner plug-in.

At this point, it should be noted that while a context plug-in is simply required to implement the IContextPlugin interface, it is usually more convenient to extend one of the helper abstract classes which provide default code for much of the required functionality. This functionality is illustrated in the following figure:





For instance, a class extending the AbstractContextPlugin can simply implement the activate() and deactivate() methods to start and stop generating events accordingly. The generated events are delegated to the context middleware via the fireContextChangedEvent() method.

On the other hand, a class extending the AbstractContextReasonerPlugin can simply implement the contextChanged() method to receive and handle context change events. The activate() and deactivate() methods are already implemented, and automatically register the context plug-in for notification of relevant context events (as defined in the XML service descriptor). (In case you need to overload the activate() and deactivate methods, make sure that you delegate the call to the super-class as well, i.e., super.activate() or super.deactivate()).

In both the cases of extending the AbstractContextPlugin and AbstractContextReasonerPlugin, the developers should define the provided (and optionally the required) context types as it is illustrated here:

<?xml version="1.0"?>

<component name="UserInTheRoom" immediate="true">

<implementation class="cy.ac.ucy.cs.osgi.context.user_in_the_room.plugin.UserInTheRoomSensor"/>

<service>
<provide interface="org.istmusic.mw.context.plugins.IContextPlugin"/>
</service>

<property name="PROVIDED_CONTEXT_TYPES" value="user-in-the-room"/>
<property name="PROVIDED_CONTEXT_TYPE_ENTITY[user-in-the-room]" value="#concepts.entities.environment|room"/>
<property name="PROVIDED_CONTEXT_TYPE_SCOPE[user-in-the-room]" value="#concepts.scopes.abstract.user_in_the_room"/>

<property name="REQUIRED_CONTEXT_TYPES" value="motion bluetooth"/>
<property name="REQUIRED_CONTEXT_TYPE_ENTITY[motion]" value="#concepts.entities.environment|room"/>
<property name="REQUIRED_CONTEXT_TYPE_SCOPE[motion]" value="#concepts.scopes.abstract.motion_detected"/>
<property name="REQUIRED_CONTEXT_TYPE_ENTITY[bluetooth]" value="#concepts.entities.device|this"/>
<property name="REQUIRED_CONTEXT_TYPE_SCOPE[bluetooth]" value="#concepts.scopes.resources.network.bluetooth"/>

</component>

In practice, you first define some aliases for each provided (or required) context type, and then you define the entity and the scope for each one of those accordingly.

Developing the User-in-the-Room Context Reasoner Plug-in

To develop the User-in-the-Room context reasoner plug-in, we also follow the steps defined in the methodology.

Step 3.1: Implement (or reuse) the corresponding context model

In this step, we need to define the model for the User-in-the-Room context type. As such, we first identify the required context values. In this case, we can simply abstract this type of information with a simple boolean. Following a process like the one described in Tutorial 7, we conclude to a model as illustrated in the following figure.





Step 3.2: Define the MANIFEST and XML descriptor of the Context Plug-in

The MANIFEST for the context reasoner plug-in is defined just like it is defined for any other OSGi component. In our case, the resulting file is illustrated in the following:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: User-in-the-room sensor context plugin
Bundle-SymbolicName: UserInTheRoomSensor
Bundle-Version: 1.0.0
Bundle-ClassPath: .
Import-Package: org.istmusic.mw.context.plugins,
org.istmusic.mw.context.events,
org.istmusic.mw.context.util.scheduler,
org.istmusic.mw.context.model.api,
org.istmusic.mw.context.model.impl,
org.istmusic.mw.context.model.impl.values,
org.istmusic.mw.context.ontologies,
cy.ac.ucy.cs.osgi.context.bluetooth.model,
cy.ac.ucy.cs.osgi.context.motion.model,
javax.swing
Export-Package: cy.ac.ucy.cs.osgi.context.user_in_the_room.model
Service-Component: OSGI-INF/UserInTheRoomSensor.xml

Next, the service descriptor is defined. As it was discuess in the previous paragraphs, this step requires that the provided and required context types are explicitly defined. In the case of the User-in-the-Room plug-in, the service descriptor was already illustrated in the previous paragraphs.

Step 3.3: Extend the abstract context sensor or context reasoner

As the User-in-the-Room has been identified as a context reasoner, we extend the AbstractContextReasonerPlugin class. The code of the UserInTheRoomSensor class is illustrated in the following:

package cy.ac.ucy.cs.osgi.context.user_in_the_room.plugin;

import org.istmusic.mw.context.plugins.AbstractContextReasonerPlugin;
import org.istmusic.mw.context.events.ContextChangedEvent;
import org.istmusic.mw.context.model.api.IContextElement;
import cy.ac.ucy.cs.osgi.context.motion.model.MotionContextElement;
import cy.ac.ucy.cs.osgi.context.bluetooth.model.BluetoothContextElement;
import cy.ac.ucy.cs.osgi.context.user_in_the_room.model.UserInTheRoomContextElement;

public class UserInTheRoomSensor extends AbstractContextReasonerPlugin
{
public static final String PLUGIN_ID = "User-in-the-room Sensor";

public static final double THRESHOLD = 0.12d;
public static final long MOTION_VALIDITY_PERIOD = 15000L; // 15 seconds

public UserInTheRoomSensor()
{
 super(PLUGIN_ID);
}

private boolean userInTheRoom = false;
private boolean userBluetoothDeviceAttached = false;

public void deactivate()
{
 userInTheRoom = false;
 userBluetoothDeviceAttached = false;
 super.deactivate();
}

private long lastMotionTimestamp = 0L;

public void contextChanged(ContextChangedEvent event)
{
 IContextElement [] contextElements = event.getContextDataset().getContextElements();

 for(int i = 0; i < contextElements.length; i++)
 {
     final IContextElement contextElement = contextElements[i];
     if(contextElement instanceof MotionContextElement)
     {
         lastMotionTimestamp = System.currentTimeMillis();
     }
     else if (contextElement instanceof BluetoothContextElement)
     {
         BluetoothContextElement bluetoothContextElement = (BluetoothContextElement) contextElement;
         userBluetoothDeviceAttached = bluetoothContextElement.contains(UserInTheRoomVisualComponent.DEFAULT_USER_DEVICE_ID);
     }
     // else ignore
 }

 final boolean motionDetectedRecently = lastMotionTimestamp + MOTION_VALIDITY_PERIOD > System.currentTimeMillis();
 if(motionDetectedRecently && userBluetoothDeviceAttached)
 {
     if(!userInTheRoom)
     {
         userInTheRoom = true;
         UserInTheRoomContextElement userInTheRoomContextElement = new UserInTheRoomContextElement(PLUGIN_ID, userInTheRoom);
         fireContextChangedEvent(this, userInTheRoomContextElement);
     }
 }
 else if(!userBluetoothDeviceAttached)
 {
     if(userInTheRoom)
     {
         userInTheRoom = false;
         UserInTheRoomContextElement userInTheRoomContextElement = new UserInTheRoomContextElement(PLUGIN_ID, userInTheRoom);
         fireContextChangedEvent(this, userInTheRoomContextElement);
     }
 }
}
}

The most interesting part of this class is the implementation of the contextChanged() method. More details about this code is provided in the following paragraphs.

Step 3.4: Implement the "activate" and "deactivate" methods to start and stop the generation of context events respectively

In this case, the two methods (activate and deactivate) are not very significant because the context plug-in is triggered by input context events arriving at the contextChanged() method (rather than timed events from a scheduler, like those defined Tutorial 8 for example).

The only noteworthy observation is that the deactivate() method is over-ridden (to initialize some local variables) but the delegation is forwarded to the super class anyway. This is important, as it was discussed already, because it allows for the automatic registration and unregistration with the context middleware.

Step 3.5: Implement the "contextChanged" method to handle received context events [optional step for context reasoners]

As it was already argued, the most interesting part of this class is in the contextChanged() method. In here, received events are analyzed to infer whether the detected motion and the Bluetooth's presence (or absense) should trigger an event. Once it is decided that an event needs to be triggered, the fireContextChangedEvent helper method is invoked.

Step 3.6: Pack everything in a JAR file, as per the OSGi’s bundle packaging specification

This is a mandatory step for every OSGi component. The process to package the plug-in has been described in detail in the previous tutorials.

This concludes the development of the User-in-the-Room context reasoner plug-in. It is assumed that the other two plug-ins are already implemented, and we resume with the development of the CaMP application.

Step 4: Develop the functional aspects of your application

This step has no significant details. Just like in usual software development, we try to reuse existing libraries as much as possible. For example, in the case of the CaMP application, we reuse the Java Media Framework (JMF) [1] libraries by SUN in order to avoid having to develop complex media handling code ourselves.

What is important to notice however, is that with this approach it is possible to build your application with Separation of Concerns [2]. At first, you develop the functional aspect of your application (the media playback code and the GUI in this case) and then you hook-up the context-aware behavior (see next step).

Step 5: Register your application to the relevant context types and implement the code that adapts the application accordingly

To bind a client to the context middleware, one need to follow these steps:

1. Define your client as an OSGi component
2. Define a service descriptor with a reference to the “IContextAccess” service
3. Implement the “IContextListener” interface and define the code that will handle the asynchronous context notifications
4. Pack everything in a JAR file, as per the OSGi’s bundle packaging specification

The first step is straight-forward. As the context middleware offers its services over OSGi, the clients must also be defined and deployed as OSGi components.

To enable automatic context access (using Declarative Services), the developer needs to define a reference to the IContextAccess service in the service descriptor. For example, consider the service descriptor of the CaMP application:

<?xml version="1.0"?>

<component name="CaMP">

 <implementation class="cy.ac.ucy.cs.camp.CaMP"/>

 <reference name="context_access"
         interface="org.istmusic.mw.context.IContextAccess"
         bind="setContextAccess"
         unbind="unsetContextAccess"
         cardinality="0..1"
         policy="dynamic"/>

</component>

As it is illustrated in this descriptor, the IContextAccess service is referenced and appropriate binding methods are defined. The service descriptor is of course referenced by a Manifest file, as illustrated in the following figure:



Finally, the actual code where the binding to the context middleware and the handling of the context events takes place is illustrated here in the following:

package cy.ac.ucy.cs.camp;

import org.osgi.service.component.ComponentContext;
import org.istmusic.mw.context.IContextAccess;
import org.istmusic.mw.context.plugins.IContextPlugin;
import org.istmusic.mw.context.exceptions.ContextException;
import org.istmusic.mw.context.events.IContextListener;
import org.istmusic.mw.context.events.ContextChangedEvent;
import org.istmusic.mw.context.ontologies.DefaultMusicOntologyV0_1;
import org.istmusic.mw.context.model.api.IEntity;
import org.istmusic.mw.context.model.api.IScope;
import org.istmusic.mw.context.model.api.IContextElement;
import org.istmusic.mw.context.model.api.IValue;
import org.istmusic.mw.context.model.impl.values.BooleanValue;

import javax.swing.*;
import java.util.logging.Logger;
import java.awt.*;

public class CaMP extends JFrame implements IContextListener
{
public static final String DISCONNECTED = "CaMP - Disconnected from the context middleware";
public static final String CONNECTED = "CaMP - Connected to the context middleware";

public static final IEntity ENTITY = DefaultMusicOntologyV0_1.ENTITY_ENVIRONMENT_ROOM;
public static final IScope SCOPE = DefaultMusicOntologyV0_1.SCOPE_ABSTRACT_USER_IN_THE_ROOM;

private final Logger logger = Logger.getLogger(CaMP.class.getCanonicalName());

private final MediaPlayer mediaPlayer;
private final UnderTheHoodPanel underTheHoodPanel;
private final AboutPanel aboutPanel;

public CaMP()
{
    super("CaMP");

    setLayout(new BorderLayout());
    final JTabbedPane tabbedPane = new JTabbedPane();
    add(tabbedPane, BorderLayout.CENTER);

    mediaPlayer = new MediaPlayer();
    tabbedPane.add("Media player", mediaPlayer.getVisualComponent());

    underTheHoodPanel = new UnderTheHoodPanel();
    tabbedPane.add("Under the hood", new JScrollPane(underTheHoodPanel));

    aboutPanel = new AboutPanel();
    tabbedPane.add("About", aboutPanel);

    pack();
    tabbedPane.updateUI();
    setSize(400, 300);
    setVisible(false);
}

protected void activate(ComponentContext componentContext)
{
    logger.info("CaMP: activate");

    setVisible(true);
}

protected void deactivate(ComponentContext componentContext)
{
    logger.info("CaMP: deactivate");

    // make sure the media player is stopped
    mediaPlayer.stop();

    setVisible(false);
}

public void setContextAccess(IContextAccess contextAccess)
{
    logger.info("CaMP: setContextAccess");

    try
    {
        contextAccess.addContextListener(ENTITY, SCOPE, this);

        setTitle(CONNECTED);
    }
    catch (ContextException ce)
    {
        logger.severe(ce.getMessage());
    }
}

public void unsetContextAccess(IContextAccess contextAccess)
{
    logger.info("CaMP: unsetContextAccess");

    try
    {
        contextAccess.removeContextListener(ENTITY, SCOPE, this);

        setTitle(DISCONNECTED);
    }
    catch (ContextException ce)
    {
        logger.severe(ce.getMessage());
    }
}

public void contextChanged(ContextChangedEvent event)
{
    final IContextElement[] contextElements = event.getContextDataset().getContextElements();
    for(int i = 0; i < contextElements.length; i++)
    {
        final IContextElement contextElement = contextElements[i];
        IValue value = contextElement.getContextData().getContextValue(
                DefaultMusicOntologyV0_1.SCOPE_ABSTRACT_USER_IN_THE_ROOM).getValue();
        BooleanValue booleanValue = (BooleanValue) value;
        if(booleanValue.getBooleanValue().booleanValue())
        {
                mediaPlayer.start();
         }
        else
        {
                mediaPlayer.stop();
         }
    }
}
}

Just like in the context reasoner plug-ins, the most interesting part here is the implementation of the contextChanged method, defined in the IContextListener interface.

The actual context-aware logic is very simple: The received events are analyzed and when an event corresponds to the user entering the room, then the media playback is resumed. Otherwise, if the received event corresponds to the user exiting the room, then the media playback is paused.

Step 6: Pack your application in JAR file as per OSGi's bundle packaging specification

Of course, once the code and the artifacts are finalized, they are all compiled and packaged together in a JAR file, according to the OSGi's packaging specification.

JARs of the used plug-ins, including the source code and an ANT build script, are available here:

• CaMP.jar
• BluetoothSensor.jar
• MotionSensor.jar
• UserInTheRoomSensor.jar
• JMFBundle.jar (needed by the CaMP and the MotionSensor bundles)

Exact details about the requirements and the deployment method for CaMP are provided in this webpage. Finally, some screenshots of the CaMP application active are available here (click to enlarge):

References

[1]. SUN Microsystems, Java Media Framework (JMF), http://java.sun.com/javase/technologies/desktop/media/jmf

[2]. Nearchos Paspallis, Frank Eliassen, Svein Hallsteinsen, and George A. Papadopoulos, Developing Self-Adaptive Mobile Applications and Services with Separation-of-Concerns, At Your Service: Service-Oriented Computing from an EU Perspective, E. Di Nitto, A-M. Sassen, O. Traverso and A. Zwegers (eds), MIT Press, June 2009, chapter 6, pp. 129-158