2011 24th SIBGRAPI Conference on Graphics, Patterns, and Images Tutorials
Adessowiki: Collaborative Scientific Programming Environment Leticia Rittner∗ , Andr´e V. Sa´ude† , Alexandre G. Silva‡ , Rubens C. Machado§ , Mariana P. Bento∗ , and Roberto A. Lotufo∗ ∗ School
of Electrical and Computer Engineering State University of Campinas, Campinas, Brazil Email:
[email protected],
[email protected] † Federal University of Lavras Email:
[email protected]fla.br ‡ Santa Catarina State University Email:
[email protected] § Center for Information Technology Renato Archer, Campinas, Brazil Email:
[email protected]
For the image processing area, the benefits of Adessowiki as a collaborative environment can be observed with no difficulties, and the three years of experience with the system has shown such benefits in practice. Possible applications of Adessowiki are: teaching environment, where students submit their reports and exercises on Adessowiki without needing to install special software; supporting the collaborative writing of scientific texts, such as papers, technical reports, and books, in which their figures are generated in a reproducible way by programs written by the authors; collaborative research environment, allowing multiple researchers of a group to exchange scientific results along with the algorithms implementation; comparison of solutions and benchmarking of algorithms given that all the programs are executed under the same configuration; prevention of plagiarism due to logging capabilities; automatic conversion to different document and slide show formats (HTML, PDF, Latex); creation of an encyclopedia of algorithms and executable source code solving problems with the same data set, easing the task of comparisons between them.
Abstract—Adessowiki is a collaborative environment for teaching and research in image processing. Adessowiki is composed of a collection of collaborative web pages in the form of a wiki. The articles of this wiki can embed programming code that will be executed on the server when the page is rendered, incorporating the results as figures, texts and tables on the document. The integrated collaborative environment of Adessowiki, containing documentation, programming code and execution results is able to create several possibilities of applications. This paper presents some of the applications where Adessowiki has been used, such as Scientific Writing and Virtual Learning Environment. Keywords-collaborative programming; software engineering; image processing teaching;
I. I NTRODUCTION The Adessowiki environment is available at http://www. adessowiki.org and its main objective is the collaborative development of numerical programming algorithms along with its documentation or scientific and technical report. The system is composed of a collection of editable web pages forming a wiki. What makes Adessowiki different from other wikis is the possibility of its pages to embed source code that is executed when rendering the page. And because texts, images, equations, graphics, plots, diagrams and tables that are incorporated to the document are generated by the embedded code, it is easy to verify if the code works. Also, the portions of code are executed on the server, allowing centralization of hardware and software configuration, and freeing the user to develop its applications only using a web client, under a Personal Computer (PC), Personal Digital Assistants (PDA), tablets or even browser enabled cellphones. The combination of a collaborative wiki environment, central server with software and data and execution of source code at render time enables an unnumbered possibilities of applications. This project is focused on the image processing area, as it is the specialty of the research team, however it may be used on nearly any area of scientific computing and engineering. 978-0-7695-4549-3/11 $26.00 © 2011 IEEE DOI 10.1109/SIBGRAPI-T.2011.12
II. A DESSOWIKI OVERVIEW Adessowiki is a distributed system, based on the technologies and concepts of Web 2.0, designed to facilitate the collaborative creation of scientific content. It is a kind of collaborative literate programming [1] powered by the modern world-wide web. What makes Adessowiki special in relation to other wikis and Web applications is the ability to include parts of code that are executed when rendering the wiki documents, in such a way that the results of the computation may become part of the rendered documents, in the form of figures, texts and tables. These code fragments are executed in a centralized host, with pre-configured hardware and software. Another advantage of this arrangement is that the user only needs a web client and a web connection to develop and test his application. 56
Figure 1 shows the computational organization of the Adessowiki. The system is composed of two separated web servers: the wiki server, that serves information from a database and never access the local file system, and the media server, which serves files from the local file system. The main components of the Adessowiki are the wiki itself, the wiki markup language, reST, and the Execution Sandbox (XSandbox), responsible for the execution of code fragments in a controlled environment. The sandbox executes in the same host computer as the media server, in such a way that the media created by the execution are made visible in the web by the server.
Adessowiki has a built-in access control system, with permissions for users and groups. As exposed previously, permissions are specified by space, whereas users and groups themselves are global information accessible to all the spaces. Registering of new users is done through an invitation system. An invitation is an encrypted text that specifies user permissions. It is created by an Adessowiki administrator and sent to the users. The registration system validates the user through its e-mail before activating its account. The wiki implementation is based on the Model-ViewController (MVC) pattern and uses an Object-Relational Mapping (ORM) to work with its database. The implementation language is Python and the supporting web framework is the Django [3]. Some features of the wiki are: • Tags. Tags are a powerful tool for content organization. Instead of filing a page under a specific folder for classification purposes, wikis allow users to associate tags to the page, and provide tools for searching pages that contain a specific tag or a combination of them. One prominent example of content organizing through tags is the current version of Gmail, the Google webmail, where all mail is stored together and tags differ between inbox, spam, etc. • Comments. Comments allow users not only to edit, but to expose opinions on a more systematic and organized way on the pages. • Alerts. Alerts of changes through RSS notifications as well as email automatic messages allow users to look up for the work on the desired selected pages. • Django. The Django MVC framework is based on Python, a programming language whose use is growing due to its power and easiness of use.
:HE
Fig. 1. System organization composed of main server (wiki database) and media server (file system).
B. The Markup Language
A. The Wiki Like other wikis, Adessowiki is composed by a collection of hypertext pages without any structure except those created by the links. This is one of the main points of a wiki: in contrast to most content managing systems, a wiki has no implicit hierarchical structure. Besides its software development features, Adessowiki may be characterized as a modern and fully-featured wiki [2]. Adessowiki is formed by a set of spaces, that ultimately are individual wikis that share administrative information, like users, groups, etc. Corporative wikis in general use these spaces to ease the management of multiple wikis. On Adessowiki permissions for reading and writing for users and groups are adjusted for each space. The users and groups database, however, is global. Every Adessowiki page is under version control. Changes made to the wiki pages are stored on a history of revisions such that any previous version of any page can be recovered. This feature incorporates implicitly concepts of source code version control like the Subversion1 and CVS systems.
The markup language used to create Adessowiki pages is called reST and it is based on reStructuredText, a plain text markup language widely used in the Python community. Figure 2 shows the document processing steps. 1) The user creates a reST document using the Adessowiki web interface. 2) The document is parsed to create an XML tree. In this step the Execution Sandbox is used to execute the Python code fragments embedded in the page. The resources created by this code execution, images for example, are incorporated in the XML document. 3) The XML document is transformed to create an HTML page that will be displayed by the Adessowiki web interface. 4) The same XML document can be used for the generation of other kinds of representations like PDF documents. The most important features of the Adessowiki markup language are the ones related to code execution, creation of Python modules and building of C/C++ Python extensions.
1 http://subversion.tigris.org/
57
"#
!
!
Fig. 2.
Document processing steps, including textual input by web browser, parsing, code interpretation and generation of a visualization format.
1) Execution of Python Code: The following fragment of the markup code causes the execution of the enclosed Python code: 1 2 3 4 5 6 7
environment for evaluation of Python code. The sandbox is implemented as a separate process that shares the same file system as a media server. The execution environment enforces security restrictions for the code that will be executed. The code can not: 1) write to the file system (reading from some locations is allowed, so that uploaded files are available); 2) open sockets or access another host directly; 3) spawn processes or threads; 4) make other system calls, such as signals. In order to be resilient to bugs eventually present in the user code to be executed, it is important for the server to be fork-based (as opposed to threaded servers). The use case is as follows. When the wiki starts the parsing of a page, it establishes a connection to the sandbox. The sandbox forks a process to provide the execution service. The code fragments are executed in this environment (so that the state is kept between fragments). When the parsing finishes, the wiki closes the connection and the forked process terminates. The images and other resources created by the execution are stored in the local file system, that is shared with the media web server. The modules created by the sandbox are also kept in the file system as Python packages. These packages are registered in the Python path and can be imported by the pages that have access to the current namespace.
.. code:: python :img_width: 320 thresh = 20 a = mmreadgray(’sign.jpg’) b = mmgradm(mmopen(a, mmsebox(2))) mmshow(a, b>thresh, title=’Contour’)
This simple example reads an image (line 5), process it (line 6) and shows the resulting image (line 7). The function mmshow is overridden by the sandbox execution environment so that the image is captured and made available through the Adessowiki media server. For another example of the source markup and the resulting HTML page see Figure 3. 2) Creation of Python Modules: To enable code reuse, the Python code contained in a page can generate a Python module that can subsequently be imported on other pages. The identification of the page and the enclosing space are used to name the module: the namespace name is the Python package and the page name identifies the module. 3) Building C/C++ Python Extensions: Another useful capability of the Adessowiki is the automatic creation of C/C++ extensions for Python. The user embeds C/C++ code in the page, identify the functions that are to be interfaced to Python and the sandbox creates a compiled Python module that can be used in other pages.
III. E XPERIENCES AND E VALUATION The Adessowiki system is achieving three years of experience as an educational environment and as a collaborative tool for writing articles and other scientific texts. Due to the image processing tools currently available in the system, the experience of teaching, learning and collaborative document
C. The Execution Sandbox The code fragments embedded in a wiki page are executed through the Execution Sandbox (XSandbox): a controlled
58
!"# $$ % &' %()#*+%()# ,-./'01/2&0,-./3'01/ &0/(3#+%()3#0
4+ 256& 76&$ 8 $$ )+ -+ 4 ) -+ 4 ) -+ 4.- ) . ) .) 9( +: 5( 7 $-+ ; ( +$-+ 2 5 6 & ; 7 6 & & +-
