09-10 Interaction Design Lab 1 Final Projects

The three members of the group did programming for Fidelio and each one developed a specific part. The programming was divided in the following areas: the general function and logic of the program, the wardrobe function and the camera function. The code at the end was put together and it resulted in one of the most challenging parts of the project.

The main code was divided in three different tabs: animation, logic and graphics. In the animation tab a code developed by MasQuest was used to set the movement of the menu. The logic tab manages the actions, such as calling each menu and screens. The graphic tab loads and unloads the images and holds the functions that are called by the logic tab.

The different screens in Fidelio are ordered in the logic section of the code. Each one of these screens holds different sub screens. This hierarchy was ordered by placing all the main screens in one section.

// SCREENS

int SCREENSETUP = 0;

int SCREENCAMERA = 1;

int SCREENMENU = 2;

int SCREEN_PIDGEON = 3;

int SCREENPROFILE = 4;

int SCREENPIAZZA = 5;

int SCREENADDGARMENT = 6;

int SCREENHISTORY = 7;

int SCREENPIAZZAUSERS = 10;

int SCREENUSERSINSIDE1 = 11;

int SCREENMARKERANIMATION = 12;

int SCREEN_WARDROBE_SELECT = 13;

int SCREEN_WARDROBE_LOOK = 14;

int SCREEN_WARDROBE_LOOK2 = 15;

int SCREEN_EVALUATION = 16;

int SCREEN_DAY= 17;

int SCREEN_WORK = 18;

int SCREEN_SAVANT = 19;

int SCREEN_WARDROBE_LOOK_REST = 20;

int SCREEN_WARDROBE_LOOK_HAT = 21;

The sub screens were separated in groups corresponding to the main screen it belongs to. For example the main screen of the piazza (SCREENPIAZZA) had the following sub screens:

int PIAZZA_OPTION_ALTUS = 0;

int PIAZZA_OPTION_PAPEUR = 1;

int PIAZZA_OPTION_OPULENS = 2;

This arrangement gave order to the code and it made it much easier to find the corresponding function and graphics when a change needed to be made.

The following line of codes specifies the initial screen for each section. For example piazza_option_selected calls PIAZZA_OPTION_ALTUS because it’s the first screen seen when the user enters the Piazza section.

// Enters initial screen for the specific section

int menu_option_selected = MENU_OPTION_WARDROBE;

int piazza_option_selected = PIAZZA_OPTION_ALTUS;

int users_option_selected = USERS_OPTION_NICOLA;

int users_selected = USERS_SELECTED_KNIFE;

int users_vandalize = SCREENUSERSVANDALIZE;

int garment_option_selected = ADD_GARMENT_ADD;

int wardrobe_section = WARDROBE_SECTION_HAT;

int wardrobe_hat_selected = WARDROBE_HAT1;

int wardrobe_shirt_selected = WARDROBE_SHIRT1;

int wardrobe_tie_selected = WARDROBE_TIE1;

int wardrobe_jacket_selected = WARDROBE_JACKET1;

int wardrobe_pants_selected = WARDROBE_PANTS1;

int wardrobe_shoes_selected = WARDROBE_SHOES1;

int wardrobe_evaluation = WARDROBE_EVALUATION;

int wardrobe_day = WARDROBE_EV_DAY;

int wardrobe_work = WARDROBE_EV_WORK;

int savant_message = SAVANT_COMPLETE;

int wardrobe_look = WARDROBE_LOOK_1;

int wardrobe_look2 = WARDROBE_LOOK_2;

int pidgeon_message = PIDGEON;

int wardrobe_look_rest = WARDROBE_LOOK_REST;

int wardrobe_look_hat = WARDROBE_LOOK_HAT;

This function manages each sub screen during an action. The following piece of code illustrates the example:

else if (piazza_option_selected == PIAZZA_OPTION_ALTUS) {

if (keyCode == LEFT){

piazza_option_selected = PIAZZA_OPTION_OPULENS;

The graphics tab besides holding images controls the unloading function of the program. It was necessary to manage the memory this way because otherwise the program is too heavy for a mobile phone to handle. Creating a general unloading function sets the first part of this function:

void unloadAll(){

unloadMenu();

unloadPiazzaScreen();

unloadPiazzaUsersScreen();

unloadUsersInsideScreen();

unloadUserVandalizeScreen();

unloadScanGarmentScreen ();

unloadHistoryScreen();

unloadMenuCamera();

unloadPidgeon();

unloadWardrobeEvaluation ();

unloadWardrobeDay ();

unloadWardrobeWork ();

unloadSavantScreen ();

unloadWardrobeLook2 ();

unloadWardrobeLookRest ();

unloadWardrobeLookHat ();

unloadWardrobeLook ();

unloadWardrobeScreen ();

unloadProfileScreen ();

Runtime.getRuntime().gc(); //Gargabe collector (frees memory)

The code line Runtime.getRuntime().gc(); //Gargabe collector (frees memory) is a java patch that frees memory by collecting unused elements.

The next part of the unloading function is created by setting individual loading stages for each screen. Note that the example also contains the unloading function of the screen. In this case the images are loaded but if there was another screen being called the unload function of the Wardrobe would be called.

void loadWardrobeEvaluation (){

evaluation = loadImage("EVAL_or_POSS.png");

poss = loadImage ("poss.png");

void unloadWardrobeEvaluation (){

evaluation = null;

poss = null;

(Runtime.getRuntime()).gc();

void enterWardrobeEvaluation(){

unloadAll();

loadWardrobeEvaluation ();

screenMode = SCREEN_EVALUATION;

y = 640;

void drawWardrobeEvaluation(){

y = easeOut(y, yTarget, 8);

if (wardrobe_evaluation == WARDROBE_EVALUATION){

image(evaluation,0,y);

else if (wardrobe_evaluation == WARDROBE_POSSIBILITIES){

image(poss,0,y);

The enterWardrobeEvaluation function unloads every image except for the ones needed for the Wardrobe Evaluation Screen. This function is repeated in each section so the mobile phone uses memory only for a small number of images instead of the whole set of images that are used in the prototype.

As mentioned before, the animation tab controls the movement of each screen. The code is called with the function easeOut.

y = easeOut(y, yTarget, 8);

The wardrobe screen uses a mask to show an animation in only one part of the screen. The maskSelect function manages the placement of the image of each piece of garment. It’s set at a specific initial value that is changed every time a new piece of garment is shown. The function wardrobe_xTarget sets the horizontal position of the garment. The following example illustrates the function:

if (wardrobe_section == WARDROBE_SECTION_HAT){

wardrobe_yTarget = 0;

maskSelect=20;

if(wardrobe_hat_selected == WARDROBE_HAT1){

wardrobe_xTarget = 0;

else if(wardrobe_hat_selected == WARDROBE_HAT2){

wardrobe_xTarget = 240;

else if(wardrobe_hat_selected == WARDROBE_HAT3){

wardrobe_xTarget = 480;

else if(wardrobe_section == WARDROBE_SECTION_SHIRT){

wardrobe_yTarget = 320;

maskSelect=60;

if(wardrobe_shirt_selected == WARDROBE_SHIRT1){

wardrobe_xTarget = 0;

else if(wardrobe_shirt_selected == WARDROBE_SHIRT2){

wardrobe_xTarget = 240;

else if(wardrobe_shirt_selected == WARDROBE_SHIRT3){

wardrobe_xTarget = 480;

The wardrobe section was done with two images: the static screen and the animated garments. Each image is set in order specifically to create the moving wardrobe effect.

image(wardrobe_all, wardrobe_x+20, wardrobe_y+85, 160, 160, 20,(height/2)-75);

image(wardrobe_selected, 0,0, width, maskSelect, 0,0);

In one of the screens a frame-by-frame animation is needed. This was done using 15 frames using the Processing’s maximum speed of 30 frames per second.

PImage mfirst;

int numFrames = 15;

int frame = 0;

PImage[] images = new PImage [numFrames];

void loadUserVandalizeScreen (){

mfirst = loadImage("mfirst.png");

framerate(30);

images[0] = loadImage("m1.png");

images[1] = loadImage("m2.png");

images[2] = loadImage("m3.png");

images[3] = loadImage("m4.png");

images[4] = loadImage("m5.png");

images[5] = loadImage("m6.png");

images[6] = loadImage("m7.png");

images[7] = loadImage("m8.png");

images[8] = loadImage("m9.png");

images[9] = loadImage("m10.png");

images[10] = loadImage("m11.png");

images[11] = loadImage("m12.png");

images[12] = loadImage("m13.png");

images[13] = loadImage("m14.png");

images[14] = loadImage("m15.png");

The animation was set using the following:

frame++;

if (frame == numFrames){

frame = 14;

image(images[frame],0,0);

The camera prototype was done in another sketch. We were able to create the basic prototype for the purpose we needed. The program accesses the camera mode and takes the initial picture the user will later use in the wardrobe section to dress himself. The code works as a demo. The user takes a picture and sees himself with the final outfit on. Our original idea is that the user takes a picture and is able to see his progress. For example, if he has selected only the hat, he will only see the hat placed in his picture.

For this prototype the video and image2 libraries were used. A code provided by Nicholas Zambetti placed in a tab called image_scale makes the picture taken by the camera full screen.

The first time we ran the code it didn’t work on the Nokia Xpress Music 5310. We later found out that a variation needs to be done. Francis Li, who developed Mobile Processing, made available the code we needed. So originally the code we used looked like this:

void setup() {
 cap = new Capture(this);

and we changed it to this:

void setup() {
 cap = new Capture(this, "capture://image");

The code changed makes possible setting up the camera.

The capturing of the picture taken by the camera was done using the following:

if (captureMode == CAPTUREMODE_CAMERA) {

// capture a frame and load it into a PImage

byte[] data = cap.read();

cap.hide();

personPhoto = loadImage(data);

captureMode = CAPTUREMODE_DISPLAY;

// in display mode, we need to loop through the draw() function

loop();

else if (captureMode == CAPTUREMODE_DISPLAY) {

if(keyCode == FIRE){

enterloadLook();

The following function sends the user back to camera mode once the picture is taken (its useful if the user desires to take another picture because he didn’t like how it came out).

else{

// In display mode, pressing a key goes back to capture mode

noLoop();

cap.show(0, 0, width, height);

image(silouette, 0, 0);

captureMode = CAPTUREMODE_CAMERA;

The image taken by the camera would be below the outfit picture. The outfit picture has specific spots that are transparent so the camera picture is shown.

else if (captureMode == CAPTUREMODE_DISPLAY) {

// In display mode, draw the last captured picture.

image(personPhoto, 0, 0, width, height);

image (silouette,0,0);

image(save_button,0,0);

It’s important to point out that originally we wanted the silhouette to appear when the phone was in camera mode; however, this was not possible. Placing an image while the phone is on camera mode causes flickering. We did some research and find out that the mobile processing reference describes the problem:

Note that, if you are drawing frames of animation while the viewfinder is showing, you may overwrite the viewfinder and cause flickering on the display.

The main problem this caused was that the picture had to be taken without a guide. The user needs to place himself on a specific spot so his image matches that of the outfits. Without the guide it’s more difficult to achieve.

____________________________________________________________________________________________________

To download the prototype you’ll need a mobile device with a 320 x 240 resolution, java compatible and with a camera incorporated.

Fidelio Prototype (no camera) | Download Jar Zipped File