On 29^th of March I had the pleasure of hosting a presentation, a quick “Introduction to Silverlight”. It was held at Avid branch in Kaiserslautern, in which I am proud to be Senior Software Engineer. The company’s culture promotes in-house training and I took the challenge in my hands.

Given that the allotted time was only two hours, the topics presented were just a few. Maybe in the future there will be again 8 hours workshops like I did while part of the Microsoft Student Partners Team! Since almost no one had experience with Silverlight, I have presented the history, stepping through each version of Silverlight and discover its novelties and added features. We had also some examples, already built applications, to showcase what one can do with this framework – from line of business to games, interior design scenes, deepzoom showcase.

Then, we examined each template that comes by default with Visual Studio 2010, analyzing and understanding their structure. We covered the particularities of Silverlight as a clientside plugin, asyncronous communication, xap generation, navigation framework, deep linking, entity framework and DTOs (data transfer objects), binding mechanisms. There were demos had hands-on coding, but this time only from my side – I believe that one technical presentation should always be performed actively with the audience, involving them to write code and learn valuable lessons while they code.

This was quite a marathon and my only regret was that I din’t had more time to present some additional topics and in a slower pace, to be sure that all the information presented was completely understood by my colleagues. But no worries, I expect them to pass by my office anytime with Silverlight related questions!

And below you might find useful the slides of the presentation:

Post from: Radu Poenaru's Weblog

Introduction to Silverlight

On 1^st of August 2011 I joined Avid Technology, Kaiserslautern branch as Senior Software Engineer. Another chapter in my professional life opens, now as a full time employee of an German branch of a United States company. After I finished my master and presented my master thesis, I am glad that now I will work only 8 hours per day!

What I will do there? Since I will work on the core of Interplay MAM, I will extensively use the ASP .Net framework, WCF, JQuery and Microsoft SQL. The team does not only that, but also maintains and updates additional administrative tools, so I will put in practice also Silverlight, WPF, Windows Forms skills. Not to mention that the company’s culture promotes in-house training and workshop presentations!

Post from: Radu Poenaru's Weblog

Joined Avid Technology

Yesterday I did the last step in my Media Informatics master endeavor: I presented my master thesis. After quite a few months of development on the Microsoft Surface, the application that resulted was a good start for a (possible) commercial product.

We started from the idea that a multitouch table device would offer more space, possibilities and opportunities to improve the collaboration and communication in the brainstorming phase of any project. This evolved very much during the development and conceiving the master thesis. It became a tool which not only supported and enhanced collaboration, but also, to name a few:

• offered clues gathered from Twitter and Flickr images,
• free drawing and sketching,
• text input using the virtual keyboard,
• audio recognition features,
• save / load of session,
• integration with a CRM, serving as repository;
• user-friendly navigation in sessions using also the CRM repository interface.

Post from: Radu Poenaru's Weblog

Breiny – Master thesis application

This article presents an overview of my Master thesis. It summarizes the work by presenting the contributions to the research field and also the future work based on the already created prototype. The Breiny application is meant to support the Brainstorming process by using computers in a modern way – Natural User Interface. After creating the prototype, a User Evaluation was used to prove or disprove the theoretical hypotheses.

In the Related work is presented the relevant work on which my master thesis relyies on and the systems that are the basis for comparison with the developed prototype. It considers several already available collaboration and brainstorming systems and compares their features with the proposed Framework for analysis. As a conclusion of the chapter, none of the considered systems fulfill all the requirements from the proposed Framework.

Next article proposes a conceptual approach to achieve the goals set by the proposed Framework and discusses the architectural view of the prototype. Then, the general usecase is presented with the additional case in which small mobile devices can connect to the proposed architecture. The desired set of features are transposed into actors and their usecases and analyzed.  The chapter continues with the requirements elicitation, establishing a list of Functional and Non Functional requirements that should be implemented. It also takes into consideration the existing software development patterns that will ensure that the prototype meet the desired characteristics and ensure that the software produced is flexible, easy to extend, and maintainable.

In  Implementation the actual development of the prototype is presented. The challenges of this part of the paper started with the documentation of the framework, which was not always enough, especially that the hardware chosen was very new on the market when the development of the prototype started. Then, new challenges appeared when designing the User Interface, but they were solved through several DIA cycles. Each cycle’s Analysis stage, which was done with colleagues and the supervising Professor, helped in achieving the current state of the prototype. Another challenge was the transition from synchronous internet communication to asynchronous, required because in case of slow network connection the User Interface frozen for few seconds during updates. By moving to the MVVM pattern and asynchronous communication, this problem was solved.

In  Evaluation were presented the hypotheses that governed the development of the prototype and which were extracted from the Framework of Analisys. The prototype evaluation proved to be a real success, almost all hypotheses were proved true by the participants in Evaluation Sessions. This suggest that in a near future, when more researcher will get in contact with the prototype, the adoption will gradually raise and the user’s feedback will improve the knowledge and validate as true also the non-functional requirements (like responsiveness, maintainability, flexibility, extensibility etc.).

The novelty of this application is given by some of its key features, none of which implemented in the other systems proposed as a reference:

1. Integration with other systems to provide helpers for idea generation – Twitter and Flickr;
2. The ability to easily share the outcome of the Brainstorming session through use of integration with other applications;
3. Addressing the physical limitation of the Surface by creating means for other small mobile devices to interface with the prototype.
4. More input modes – voice, drawing and typing.

These features were the novelty brought with this paperwork and the users proved that they are required, desired and usable by validating them in the User Evaluation questionnaire.

Future work

During the evaluation phase, the users’s feedback was great in possible new features and interaction chances. They proposed new features which can be implemented on the prototype, which are presented below.
Larger screen / better resolution
The feature mostly desired by the users was to have either more space at their disposal or a bigger screen resolution. Because the system is a closed unit, it cannot be upgraded, and therefore this is possible with this device. Nevertheless, the second version of Microsoft Surface will have a diagonal of 40 in (108cm) instead of 30 inch (76cm) and the resolution increased from 1024×768 pixels to 1920×1080 pixels – full HD definition. In the my opinion, this will overcome the current situation.
More virtual keyboards
The second most important improvement would be to offer more keyboards at one time – but would this suffice? More software keyboards will occupy more screen space and make unavailable the content underneath them. This is a question to which a future thesis will hopefully respond.
Better microphone system
The voice recognition system used is the default Speech Recognition implemented in Windows Vista. It requires training in order to improve its recognition rate. In my opinion, training it extensively with more subjects would improve immensely the recognition rate. Possible is that there are other speech recognition engines available which can be better than the default one proposed by this paper.
Delete session
Another feature requested by the users was to be able not only to save and load session, but also to delete it. In the my opinion, since deleting folders is supported by the BSCW system, it is better and safer to be done from it. Safer because during the session, an accidental touch from many participants who interact simultaneously can destroy a valuable resource as a Brainstorming Session. But this can be part of a future master thesis – is might be valuable to bring more features from BSCW in Breiny, such that the users may use the prototype as a limited interface to BSCW?
Delete Brainstorming Item
This feature was proposed during the User Evaluation and can be proven useful. But it conflicts with the main goal of the Brainstorming process – don’t judge, create as many ideas as possible. I believe that if a staging mechanism would be implemented, to clearly separate the Adding/Editing phase from the Grouping/Relating, this can prove useful if this feature would be enabled only in the second phase, where some ideas which don’t relate with the main goal can be removed on a common agreement between the Brainstorming Session participants.

Post from: Radu Poenaru's Weblog

Master thesis – Conclusions

The Users for the first session were 5 males and 1 female, with age ranging from 27 to 49. For the second, there were 4 males with age ranging from 26 to 52, with significant Brainstorming sessions experience, using Mind-mapping software, whiteboard computer based systems. The room setup consisted from a large space around Surface, with chairs surrounding it. All users were in the Contributor role and they had the opportunity to switch to Session Master any time. The application was started, with no Brainstorming Ideas on the tabletop.

The Evaluation session started with the Prototype presentation. This was targeted only on the User Interfaces, explaining how and which actions can be performed with each area of interest (UserControls and Main SurfaceWindow from Implementation Chapter). The users were encouraged to experience the interface and ask questions during the presentation, in order to understand better the interaction with the prototype.

Following the presentation phase was designed as a game, in order to exercise the prototype’s interface and evaluate the system and its features. The game was built around a imaginary, newly created, travel agency. The participants should create the company’s portfolio in a Brainstorming session, composed from the four phases (scenarios) presented in Typical use cases. They were disclosed one by one, in order to not affect the brainstorming process.

In the next stage, the users were supposed to group them into three categories by using the Prototype’s feature, structuring the imaginary company’s portfolio in three categories. They were encouraged to discuss about their decisions with the other users from the Brainstorming Session.

The final step was to relate them in logical structures by using the Prototype’s built-in Relating feature. The users were asked to create the structure of offered holidays.

The author of this paper was also the organizer of the User Evaluation session, observing the group dynamics, helping and reminding users the steps of specific interactions. The time of the session was limited to 45 minutes, from which the biggest share of time was allocated to the Brainstorming Item generation. The session concluded with the filling of the Questionnaire.

Hypothesis and methodology

The target of the first test was towards answering the following questions, which are the initial study goals, mentioned in Introduction:

1. Q1.   How was the User Experience using brainstorming applications running on multiuser multitouch tabletops?
2. Q2.   Is it useful to have post-brainstorming ideas organization?
3. Q3.   Is the brainstorming process successfully supported by mashing with other systems (e.g. Twitter and Flickr)?
4. Q4.   Is integration with small mobile devices a good solution to address the physical limitation of the tabletop?
5. Q5.   How important is to be able to persist the result of the brainstorming session?

By focusing on these ideas, the users will provide insight in the proposed process and also prove if the presented Breiny prototype is usable in real case scenarios. The hypotheses that were established before the process begun are presented below and state the author’s expectations from the prototype, expectations which can be proved or disproved by the Evaluation Session users:

1. H1.    Using tabletop improves the collaboration and communication among users
2. H2.    Multiple edit modes can improve creation of ideas
3. H3.    Grouping and Relating are important to structure the Brainstorming items
4. H4.    Fresh ideas from external sources improve the Brainstorming process
5. H5.    Interfacing with additional mobile devices improves the idea generation process
6. H6.    Sharing the outcome of the session is a useful feature

The questionnaire mentioned above was given to the Brainstorming Session participants and consisted from 7 pages for the first one and 8 pages for the second one. It started with creating a baseline of the user’s profile in brainstorming experience, tools used and familiarity with the tabletop used, Microsoft Surface. The next sections focused on the session itself, the social experience it brings by sharing a device, the input, grouping and relating possibilities offered by the Breiny prototype; following was the evaluation of external sources usability. For the second questionnaire, the next section was evaluating the impact of using additional devices in collaboration with the tabletop. The questionnaire ended with the evaluation of importance of sharing and saving features of the prototype and the final overview from the usability, utility and overall experience of the Breiny prototype.

Between the two User Evaluation sessions, several modifications were made, based on the user’s feedback. The most noticeable was the temporary deactivation of the Microphone Edit mode as the users felt that the session’s discussions were impeding using this device and several modifications to the mobile clients to accommodate new features offered by the prototype.

Conclusions

The following discussion is based on the results presented in Appendix A: User Evaluation Questionnaire, where the Brainstorming Session users’ feedback is presented.  The questionnaire is focused on getting answers to the research questions and the hypothesis formulated in the previous section.

As presented in the beginning of the User Evaluation with Users in this section, the group can be considered balanced.  All the questions presented to them are closed questions:

• grading difficulty types, in which the users must give a grade on a scale from 1 (very easy) to 5(very difficult);
• yes-no questions, to verify subjective opinions;
• free text areas, to allow the users to express their opinions or suggestions directly;

The first set of questions were targeted to the social, communication and collaboration side of the process, 90% of the users declaring that their movements were not impeded, the personal space was respected and they were confortable during the Brainstorming Session. The visibility was only 60%, the users mentioning that they focused on their idea generation and editing them, hence focusing on the tabletop and not on their colleagues. Regarding the physical limitation of the device itself, the general opinion was that it is too small to accommodate groups of 4-6 persons which directly interact with it, rather groups of 2-3 persons. Also the relative small resolution of the device’s screen (1024×768 pixels) made the participants to feel it “small”.

Judging these results, the H1 hypothesis may be considered partially true. Considering users’ feedback, the only part that can be improved and thus make the hypothesis fully true is not in the scope of this thesis, as it regards only to the hardware used – would be more useful that the tabletop’s screen with a bigger resolution, allowing users to better use the space available.

The next set of questions focused on the usability of input possibilities offered by prototype – using the software keyboard, drawing on the surface and using the microphone. From all users, the overall utility of the editing feature was considered very useful by 10 %, 60 percent – useful and 30% – moderated, concluding that this overall feature is important for the users. In the author’s opinion, it is possible that the users which rated it as moderated to be the ones more interested in the coordination, rather than in editing the items. For the questions particularly focused on the specific input device, there is a clear opinion that the users are more comfortable with the keyboard and the drawing than with the microphone. While all of them tried at least once the first two methods, only 2 of 6 tried the microphone, for the second User Evaluation the microphone was disabled. The feedback gathered from the last question in this set is pointing to the conclusion that it was difficult to use the microphone, probably because of the noise in the room.  Other participants mentioned that the tabletop’s Operating System offers just one software keyboard at one time, such that no two users can type at the same time. During the editing process, this is an important limitation, however outside the scope of the prototype built.

Following these results, clearly demonstrates that more than one input method is preferred and the H2 hypothesis is true. If the implementation would consider only the default mode to insert text, the software keyboard, the participants would have led to “keyboard fight”.

Following in the User Evaluation were questions regarding “Relating ideas” by color grouping and relating through graphs. More than half of the participants considered it useful and 33% very useful. The relating process was considered relatively simple or manageable by all users and the mean value was almost “Relatively simple”, 2.33 on a scale from 1 to 5, which can be considered a good value, given the fact that some of participants never interacted before with Microsoft Surface, everything being new for them in the proposed setup. All the participants used this features, 67% for “Few times” and the rest for “4-5 times”.

Given the fact that the brainstorming process was quick and the amount of ideas generated were 2-3 per participant and considering the previous paragraph statements, the author considers that also H3 hypothesis can be considered true.

The User Evaluation Questionnaire continued with “Generating ideas from external sources” which started with the question regarding the utility of this feature (the question was not focused on one mash, but considered together Flickr and Twitter interaction). It was considered very useful by 11% and Useful by 67%. From all participants, 33% used it few times and 22% “Probably one time”. This feature was considered relatively easy to use (on a scale from 1 to 5, one being the very easy) by half of the users and “Very easy” or “Manageable” by one participant, respectively. With a mean of value 2,33 we can concludes that  this feature is easy to use by all users, especially that some of the participants never interacted with the tabletop device, the entire system being completely new for them. The users suggested many new sources to mash with, denoting their interest for this specific feature. One user remarked that even if the feature was “Very easy to use, the result retrieved wasn’t useful”, proving the fact that more alternative sources would have been improved the Brainstorming Session.

Following the above particular conclusions, it is proven that the fourth hypothesis is true and the system provides a useful and easy way to refresh the Brainstorming Session.

For the second group, the interaction with other mobile devices was enabled. There were two devices available to be used by the four participants to the session. All four had the opportunity to use them; one participant created 4-5 ideas and other two few ideas. From all, 75% considered that it is easier to create ideas on the iPhone than on Microsoft Surface, but 100% agreed that it is easier to interact with them on the Surface. Interestingly, half of the group felt excluded from the group when using the mobile device, being more concentrated to write the idea on it than to interact with the rest of the group. Another question revealed the fact that nobody felt uncomfortable when using the mobile device, as opposed with one of six people from the first group.

Based on the facts exposed in the previous paragraph, the author concludes that hypothesis H5 is true and the Brainstorming process is improved by using additional devices. Even if the users agreed that is easier to create ideas on their own small mobile device, having full access to keyboard (no more “keyboard fight” as already presented in the evaluation of the “Multiple input possibilities”), they also fully agreed that the interaction with the already created content is enhanced on the tabletop.

The last feature to evaluate was the sharing of the Brainstorming Session, where half of the participants considered it important and 10% essential, while 40% considered it useful. When asked about if they would like to have the possibility to later review the session, 50% considered it “Very useful” and the rest “Useful”. Regarding the willingness to review it with other colleagues, again all replyied yes, from which 20% “For sure YES”, 50% “YES, most of the time” and 30% “Probably YES”.

Based on previous data, also the H6 hypothesis is confirmed true, users indeed desire to share data, have the possibility to review it and for sure reviewing it with their colleagues.

The final part of the Questionnaire was related to the usability of the prototype. Most of the participants replied that the consider it 3/10, 1/10 being the best grade, the mean of 3.2 proving that even if the prototype is usable, it still requires some improvements; some of them have been implemented and the rest are proposed as future work. From the Utility point of view, most of the users considered it helpful and again considered it 3/10, 1/10 being the best grade, the mean of 2.8; I conclude that once the usability of the prototype will be improved, also its utility will increase. The overall User experience was graded as 2/10 by almost half of the participants, 1/10 being the best grade, the mean of 3.1 meaning that the prototype is engaging and pleasant to use. Another impressive observation is that even if the proposed application is still a prototype, all the participants would recommend it to their colleagues as a Brainstorming tool.

In conclusion, the evaluation sessions revealed that almost all hypotheses were true, with only one partial true because of the limitations of hardware used. We also found some problems occurred only on multi-user environment, such as the microphone issue – because of the noise it proved not to be a very useful feature, . Also, a flaw was detected when users tried to activate and deactivate the microphone several times in a very short time span, the application crashed. These were useful feedbacks and were taken into consideration and fixed. The author is very happy with the outcome and the feedback of the User Evaluation session of the Breiny prototype and believes that this prototype is close to become a concrete application.

Post from: Radu Poenaru's Weblog

Master thesis – User evaluation

The theoretical evaluation of the thesis work starts with by assessing in which measure the research questions were answered by the proposed concept. Then, the analysis will focus on the comparison of the implemented prototype’s features with the ones presented in Related work, in order to see which ones were implemented and how the prototype’s features map on the array of desired ones. Finally, this chapter will conclude with a matrix in which the implemented functional requirement are marked, giving a final overview on the capabilities of the prototype to fulfill all the requirements stated in Requirement elicitation.

The section Concept – Solution approach gave a possible solution to the research questions, proposing several answers to them. In this section, the paper will analyze them in the context of the prototype’s implementation.

1. Additional features to improve the usability of a brainstorming supporting application;
2. Input devices / mechanisms to register ideas into the system;
3. Post brainstorming organization of ideas
4. Mechanisms to support the brainstorming session;
5. Is the brainstorming session result used by other systems?
6. Solutions to address the physical limitation of the tabletop.

The next table presents the analysis of the prototype’s features against the desired feature matrix in which all related tabletop applications which support brainstorming were already analyzed.

As the above table presents, the implemented prototype achieves all the proposed goals, while the best system (Touch & Write / LeCoOnt) scores only 62.5% of the desired features. As opposed with Touch & Write, the addition of helpers allows easy fresh ideas and inspiration during the Brainstorming process. The integration with other applications, another feature that only this prototype implements, allows easy share of the Brainstorming session with other scientists and researchers. Addressing physical limitation is another unique feature that this application brings, affording bigger teams of researcher to brainstorm at the same time.

The functional requirements proposed by this thesis for the developed prototype ensure that all the features that make this unique will be implemented. As it can be observed from the following table, all of them are present in the prototype.

The theoretical evaluation parts conclude with few observations: the prototype has all desired capabilities to improve the Brainstorming process as the feature matrix was fully checked. It can prove to be much more useful than the compared systems, having almost twice the features that the average.  The concept combines almost all useful features of electronic cooperation and collaboration, sharing and persistence, interlinking with other social and research systems and mobile devices.

Post from: Radu Poenaru's Weblog

Master thesis – Theoretical evaluation

The prototype’s architecture implementation

The architecture was already presented in the Prototype application architecture and now we’ll see the actual implementation. The prototype implementation tries to follow as close as possible the recommended architectural patterns and guidelines. To support the argumentation, a class diagram will be presented to give a visual orientation through the prototype implementation presentation.

The built prototype supporting Brainstorming Sessions on Microsoft Surface, further named Breiny, was developed as part of this thesis and uses all the above framework features. Thus, it is creating the basis for future work by being flexible in development yet fully packed with features.

Breiny’s class diagram

The class diagram was created with Visual Studio 2010 default diagramming tool and it is presented below. Its structure visually follows the MVVM pattern and it is composed from 4 main areas:

1. The main window – handles all underlying user controls, tag recognition and tag visualizations and offers the binding context to the Session object to all user controls. It represents the View from MVVM pattern. Additionally, for the tag interaction there is a visualization which is created each time a tag is placed on the screen of the Microsoft Surface. This tag allows personalized experiences for all the users registered into the system;
2. User controls – keeps concerns separated into several areas. The naming convention is uc for User Control, SV for the ScatterView and SV…Item for Scatter View Item. Each ScatterView has a User Control as template for the data bound item. This way the look of the items is customized to allow easy recognition by the users, an important feature since each item affords different actions;
3. Session – the ViewModel, aggregates, transforms and offers the objects it contains to the above controls for binding. It is composed from the main Session object which acts as a central hub of information, using the Singleton pattern. The design choice was made because the main screen binds only to this object; subsequent User Controls bind deeper into a specific collection contained the Session object (e.g. the BrainstormingItemList) and display the appropriate data. This approach, because it is just a single object through the application life, provides easy serialization and deserialization of the entire environment when the “Save session” or “Load Session” features are used.
4. The Model is represented by the means to get the information over the internet and the structures (models) in which it is stored. The communication is enabled by four modules which fill five model structures. Some communication modules have an independent way of refreshing their corresponding module – like in the case of CommWeb module which refreshes at a specific interval the collection of BrainItemType; other run only when invoked, like CommRPC module which connects to the BSCW server only when required.

Depicted with arrows in the Model area are ways by which the structures are filled with data from the communication modules.

Microsoft Surface

The hardware device that will allow software development of the prototype was chosen Microsoft Surface. The reasons behind this choice are:

• It is a complete device, which allows easy operation and minimal hardware maintenance. Other multi-touch tabletops either are custom built (which is not the point of this master thesis) or require constant maintenance if they are composed from separate components – by example moving it requires recalibration because the camera responsible to process infrared touch is not precisely in the last setup point;
• It runs Microsoft’s Windows operating system, allowing development in high performance frameworks like .Net 4.0.

• The touch framework is deeply integrated in both Operating System and Hardware and easily accessible;
• The community which develops Microsoft Surface application is big, allowing easy and fast application engineering.

The device itself is presented in the left handside picture. The screen (1) is a special polarization transparent material, allowing both projection of an image from a beamer (4) included in the unit and recording of infrared touches using infrared cameras (2). The system is composed completed by a computer (3) which processes maximum 50 simultaneous touch inputs, runs applications and shows them on the device’s screen through the beamer.

Microsoft Surface’s Operating system is Microsoft Windows Vista tightly integrated with the touch framework. It allows developers to use the native framework WPF (Windows Presentation Foundation) development in C#, Visual Basic or other .Net languages supported by Visual Studio. Nevertheless, ones can use also other frameworks like Java with custom built libraries, but this with the cost of adding another set of layers of abstraction which transforms into performance degradation.

Because of my previous experience in Visual Studio, C# and WPF I decided to use the native languages to implement the already presented prototype’s architecture.

Post from: Radu Poenaru's Weblog

Master thesis – Implementation of Breiny

This section will present the Breiny application’s user interface and the available interactions with it, features which will be evaluated both theoretically and by the users. It is composed from a main SurfaceWindow named “Main screen” in figure below, on which all the interaction occurs. Following the requirements for extensibility, modularity and flexibility, the subsequent visible objects were separated into several user controls.

In the following subsections the main screen will be presented first along with its specific interactions, then the discussion will continue with the available item types and their particular available features.

Main screen

The main window acts as an aggregator for all the interactions, providing also support for the other controls and their interactions. It offers access to the main contextual menu, to the tag recognition and visualization, offers the data context through binding against the Session singleton and, because binding is propagated to the children controls, allows subsequent binding context to all of them.

In the figure from the left, the Breiny application was started and a new Brainstorming session is created. In this specific case the Brainstorming Items Repository didn’t already contain Brainstorming Items, thus none of these appeared on the application screen. But the users may have prior access to other devices that can interface with this thesis proposed system (iPhone, iPad or Adobe Air client) as mentioned in Architecture chapter and send they initial suggestions. If so, their Brainstorming items will show on the newly created Brainstorming Session.

Being a global User Interface element, the contextual menu (named ElementMenu in Microsoft Surface SDK) already mentioned in the previous paragraph is activated by tapping on the Microsoft Surface’s Screen. If the user taps again on the screen, this time in the glowing circle that appeared (where the red finger icon is shown), a contextual menu with following actions will appear:

• New Brainstorming Item – creates an empty Brainstorming item and adds it to collection.
• Clear Tweets – clears all the items from the TweetsList from the Session object, triggering the delete all visible items of corresponding User Control from the main screen. This feature is used when all needed Tweets were used and the Brainstorming Session users need more space on the screen.
• Clear Flickr images – same behavior as for Clear Tweets, but with the mention that the deleted items are from Flickr.
• Print Screen – this feature allows quickly saving a picture of what is currently on the main screen.

Tag interaction

One of the features implemented by Microsoft Surface SDK is the possibility to recognize objects that make contact with the screen specific shapes and assign them to data structures. The Tag system is used to quickly identify already set associations between recognized Tag’s number and a data structure stored in the Operating System Registry. The data stored was linked with the pre-configured users which will perform the role of Session Master. The data structure contains among others the Name, the account on BSCW server, the account used for Brainstorming Item Repository.

As future work, the system can be extended and the tags printed also on the Research Institute employees’ badges, such that the interaction with the Brainstorming system proposed by this system to be largely improved.

When a preconfigured Tag is placed on the screen, the visualization from below figure is shown, prompting the user to insert its password. Since the server is already set and the username is retrieved from the Registry, this is the single information required for a user to be able to access the panel presented in below- right side figure, allowing him or her to perform more actions.

 

User Controls

The custom User Controls can be found in Breiny’s class diagram in the View – UserControls section. The name UserControls is used in the context of the Visual Studio 2010 development environment and represent modular code encapsulated into easily reusable controls. They are still part of the View paradigm in MVVM pattern along with the already presented Main SurfaceWindow. Worth to mention is that they are not consisting only from visual elements, but rather composed from both UI elements and their attached code behind. In order to respect the separation in the MVVM pattern, only the required code was implemented in the code behind class, the rest of the logic being implemented either in helper classes or in the Session object, which represents the ModelView paradigm in MVVM pattern.

Each UserControl is placed inside of the Main SurfaceWindow as a children control and enables a key scenario in the Breiny’s lifetime:

• ucSVBrainstorming and its children’s template ucSVItemDefault provide the visual  interface and interaction with the most important items in the prototype: the Brainstorming Items (for all the children templates the interactions and visuals will be presented in the next section – Items).
• ucSVFlickr and its children’s ucSVItemFlickr provide visual interface and interaction for the system mash-up with the Flickr website, allowing users to select, preview and manipulate them;
• ucSVTwitter and its children’s ucSVItemTwitter, as the above Flickr counterpart, provide visual interface and interaction for the system mash-up with the Twitter website, allowing users to select, preview and manipulate them;
• ucSplash is used as a splash screen, shown at the application start;
• ucBrainstTitle was inspired from the Berufsakademie-Ravensburg application (Dräger, et al. 2008): the revolving text written along a spiral will allow the Session goal to be clearly specified.

User Controls – Items

The items represent the visualization of the data items – Brainstorming Item, Flickr and Twitter item. They have associated menus for special actions that can be applied to them. This subsection will discuss in detail the role each has, how can each item be created, how purpose it server, what actions or features it offers and its lifetime.

The Brainstorming Item

It is the most important item available, as it represents the center of the Brainstorming interaction – it contains the smallest unit of a Brainstorming session. Because of this, it is the most complex object in the prototype and enables a set of key scenarios.

In figure from the left is presented the User Interface of Brainstorming item. By taping on it, the contextual menu appears and the user can access the object’s features:

• Edit – after pressing it (Figure 28), a set of controls are made available to edit the content of the item. Pressing again on Edit will toggle this mode off, hiding the controls. The active mode is marked with the yellow color (in this case the mode is Drawing). The three editing modes are:
• software keyboard on the screen,
• inserting the text via Microphone
• draw on the screen with fingers or Infrared pens. By clicking on the colored dot, the users can choose another color and pressing the eraser button will switch between drawing and erasing.
• Group – after activating this mode (Figure 29), another set of controls appear, allowing the users to change the background color of the item and thus mark it visually as belonging to a color category. Pressing again on it will toggle this mode off, hiding the controls.

Figure 28  – Edit menu with painting activated

Figure 29  – Grouping menu, now item belonging to another color coded category

• Relating allows creation of a relational structure, linking related items through lines. For this, first it must be activated and afterwards, one can drag and drop the activated Brainstorming item over another Brainstorming Item, which will result in creation of a white line linking them and representing their relation (Figure 30). To delete this relation, the user must tap once the line, transforming it into a dashed red line (Figure 31) and then tap again the line to finally delete the relation between items (Figure 32). If the second tap is not executed in 3 seconds, the line will revert to its original state, white without dashes.

Figure 30  – Related Brainstorming Items

Figure 31  – Tapping once on the relation will mark it for deletion

Figure 32  – Second tap on the relation deletes it

• Send To – when user activates this submenu, the prototype retrieves from Brainstorming Items Repository the list of all active users and adds them as children to the submenu. When one of the children is selected, the Brainstorming Item is deleted from the prototype list and sent back to the repository, this time marked to be read by the target user.
• Send back – it sends back a Brainstorming item to the user that originally created it. This feature is available when the prototype received this item from another source, e.g. from an iPhone user.

The Twitter / Flickr Item

The paper proposed itself to study the importance of using data from additional social networks, Twitter and Flickr, in order to improve the usability of the prototype. The question raised is if would it be useful to query these services when the users lack ideas or they face a blocking situation in which they require some fresh perspective on their goal. Because these two item types are closely related, this section will present them both, underlying also the differences that exist between them.

First and foremost, the data retrieval can be activated by a Session Master, after its login into the prototype using the already provided tag. Once logged in, on new tag activation, user can insert in the tag visualization window (Figure 26) a word into the textbox and press either the button with Twitter image in order to retrieve Twitter public timeline or on the button with the Flickr logo to access Flickr public images. Once retrieved, the data will be shown as in the Figures 33 and 34.

Figure 33  – the Twitter Item and its contextual menu

Figure 34  – the Flickr Item and its contextual menu

In Twitter Item, the background is the Twitter bird, and the content is split in three areas: upper left can be seen the picture of the person who created the tweet, in upper right his name and his Twitter id. Underneath is the text of the tweet.

In case of Flickr item, the background is the actual image uploaded by someone and one can identify the same three areas, upper left being the Flickr icon, upper right the picture file name, and in the underneath the  picture description.

These items are temporary items; a new search on Twitter or Flickr will delete the old items and replace them with new ones. This decision was made because the screen would get very crowded with all these items, which are not always required after the first view. The mechanism to make them persistent is available as a contextual menu which allows Brainstorming Users to “promote” them to full Brainstorming Items. This way the users can also edit, group and relate them.

When all Twitter and Flickr items which contain valuable information were transformed into Brainstorming items, users may choose to remove the unused ones by tapping on the screen and choose from the contextual menu “Clear Tweets” or “Clear Flickr images”, recovering valuable screen space for further actions.

Post from: Radu Poenaru's Weblog

Master thesis – User Interface

My thesis proposes several approaches as result of the research questions. Their validity of each approach should be verified by allowing users to interact with a prototype built to fulfill all the proposed solutions. Hence, the final users will prove the correctness of this thesis approaches. The implemented prototype will be a brainstorming supporting application.

Because my thesis studies brainstorming on a tabletop, I will choose the Microsoft Surface as the underlying hardware because it supports multitouch and multiuser experience. As development environment I will use Visual Studio 2010 and the programming will be done in C# with User Interface (UI) built in Windows Presentation Foundation (WPF) framework.

In this article I will start with a brief introduction in general details of the baseline system and the technologies implied in the creation of the prototype system. I will continue with the presentation of the architecture of the actual implemented system, which tries to follow the proposed one; the differences between the proposed system and the actual implementation will be underlined and the choices motivated. The article will continue with the class diagrams of the prototype and will end with the presentation of the User Interface, allowing a clear view over the features and the way the prototype performs in achieving the goals of the previous concept presentation.

The prototype had at its base the idea and the former work during my work in Fraunhofer FIT team. The application, called ideaPitch, was a “sink” for ideas sent from different devices (iPhone, iPad, Air desktop application) to the Microsoft Surface. The application was very basic and could load and show ideas, save a printscreen. Because was created in the early stages of Microsoft Surface’s life time, it was not so well structured to accommodate the extensive feature set proposed in this paper. The ideaPitch was used only as an inspiration; the proposed prototype has a different structure and code base, allowing it to be a more powerful approach to the same basic concept – create a proper way to share your ideas with your colleagues.

Brainstorming Session repository – BSCW

In the first phase of the research for my master thesis paper, different sharing platforms were evaluated as presented in Related Work. Among these, BSCW (Basic Support for Collaborative Work) was presented as a content manager and sharing tool, proving to be a easy and straightforward tool to support and enhance the collaboration and cooperation in Computer Supported Collaborative Workgroups. Another feature required by the prototype to be implemented in this paper was a API for communication. BSCW offers this as a webservice module which exposes an API that can receive requests and reply back using the common XML format. The documentation of the methods exposed is extensive and the data retrieved very complex for some calls. Nevertheless, the granularity of retrieved data can be set in the call, such that one can avoid retrieving huge amounts of data, instead just the parts the prototype needs.

Another reason for choosing this system is that it was already available and highly used in the Fraunhofer Institute, researchers being very familiar with it and its features, thus the prototype and the representation of its data are easier to be adopted by the community.

BSCW architecture

All above reasons contributed to the decision to use it as Brainstorming Session repository, allowing not only easy sharing and increased collaboration among teams, but also a friendly and familiar environment. In my opinion, this contributes to the improvement of the adoption of this proposed system.

After choosing the environment, the next step was to identify the structures in BSCW that can accommodate the new data saved by this thesis prototype. The decision was to create a folder (as in BSCW’s folder paradigm) with a standard name and description in which all Brainstorming Sessions for this user to be kept. This decision assured that the BSCW user’s account will be well organized and have an easy to identify entry point for viewing and sharing the sessions with other users.

Each session is represented by a subfolder having the title the one that the user chosen before saving the Brainstorming session. This subfolder has also a description which states the date and time in which the session was created.

Going deeper one level into the saved structure, the session contains at least an image, the printscreen of the application. This shows the current state of the application and its purpose is to offer a visual clue about the session, but also to host in its associated metadata the XML serialization of the object which represents that specific session. This associated data is used to restore the session to its original state at a later time, resuming the brainstorming process from where it ended.

On the same level with the Brainstorming session’s printscreen will be found any Brainstorming Items associated with it as HTML files. The choice of this file type is that it affords to easy see the saved items as they were represented on tabletop, allowing users to easily exemplify their Brainstorming session’s outcome to their colleagues.

Brainstorming sessions created in BSCW interface

Brainstorming Items repository

The Brainstorming Items repository is a PHP webservice which is connected to a MySQL database. Its purpose is to register users, persist and offer to the target user the Brainstorming items that are sent to it. It is used as storage for the Brainstorming Items for the next available Breiny session.

Post from: Radu Poenaru's Weblog

Master thesis – Implementation of Repositories

This article presents the system architecture and the underlying decisions that were taken to achieve the above requirements. The system requires a repository for sessions, a repository for Brainstorming items and a native Windows application for achieving the baseline of desired features. Thus, a distributed client – server architecture is considered, exemplified below.

Session repository architecture

The above schema represents the Session repository architecture. Its main purpose is to offer two main features:

• communication over network
• means for storing, retrieving and sharing the sessions

The server and the native application can be mixed together, but with limited sharing options. Because of this reason, we will use an already created system which not only that offers through its network module of API (Application Programming Interface) both means of storing and retrieving of the session data, but also allows advanced features of sharing the data in an already familiar interface.

Brainstorming items repository architecture

The architecture for the Brainstorming items repository is similar with the one for Session repository, but the most important feature it brings is the possibility to interface with other devices, not only with the tabletop.

Another reason for creating this secondary architecture is the fact that it easily enables an offline scenario, in which the repository server is not available because of the network availability issues. Nevertheless, the mobile devices can connect to this repository through creating a local wireless network without the need to be connected to the internet. Another advantage is that, because it’s small complexity, can (but it is not required to) be installed on the same tabletop. The offline scenario is not covered in this paper, but rather proposed as future work.

Prototype application architecture

The application conceptual architecture is straightforward, using the separation of layers into Data, Business and Presentation (using the Model-View-ViewModel pattern). This allows having a very modular, easily scalable and flexible application.

The Data layer is responsible with all network communication, hiding the details of this operation to the next layer. This permit isolating the communication in simple methods, returning objects complete with their attributes, instead of local parsing of the web retrieved result. It allows reuse of code, clean separation of concerns and very good testability.  Because of different APIs which expose different methods, it consists in several sub modules for each type of system with it interacts with.

Business layer contains the session, a global state object which encapsulates all session related objects like Brainstorming items list, Twitter of Flickr item lists etc. Its main purpose is to provide collections of objects to the UI, separating this way the internal logic from the interface.

The Presentation layer’s purpose is to consume the data made available by the Business layer and present it to the user. This separation of concerns is very effective when more than one part of the User Interface needs the same collection of data, allowing easy update and synchronization between different UI areas.

After we took a look in the architecture let’s move first to the implementation of repositories and then to the Breiny’s implementation.

Post from: Radu Poenaru's Weblog

Master thesis – Architecture