Visual Studio Team Services: Discovery

1

Visual Studio Team Services: Discovery MINH-LONG DO, RICARD GASCONS, YICHEN LU, JASON MARTYRES, and KRINAL TRIVEDI, University College London Abstract: Undergraduate Computer Science courses in the UK train students to become professional software developers. The courses do not just focus on equipping the students with the necessary technical skills, but also to ensure they know how to be a part of a software development team. Typically in the second year, students take part in a large team project, where they apply their research and technical skills to complete a complex project. Monitoring the progress of a team may be straightforward, but monitoring the progress of individuals in a team poses a challenge, especially when there is a shortage of Teaching Assistants (TA). The software that we have created allows the progress of individuals in a team to be monitored in two ways. The first way is through the distribution of one hundred points amongst team members. The higher the points assigned to an individual, the greater the contribution from the team’s perspective. The second way is to track the code they have committed to a repository through metrics. Quality code is credited with higher metrics and results in a greater score for the repository. ACM Reference format: Minh-Long Do, Ricard Gascons, Yichen Lu, Jason Martyres, and Krinal Trivedi. 2017. Visual Studio Team Services: Discovery. ACM J. Comput. Cult. Herit. 9, 4, Article 1 (February 2017), 9 pages. DOI: 0000001.0000001

1

INTRODUCTION

It is usual that students start their undergraduate studies in Computer Science with zero or minimal knowledge, and end their studies with the knowledge required of professional software engineers. Besides the effort of students, it is also important that universities structure their courses to give maximum support to the student’s learning. Teaching at the department of Computer Science at University College London (UCL) follows the Scaffolding, Discovery and Innovations (SDI) methodology. On a typical three year Computer Science undergraduate course, students engage in one of these stages each year. • Scaffolding - First year students learn the basics of programming by learning four different programming languages. To enhance their learning, students are given a set of exercises which they complete in a practical laboratory session. • Discovery - Second year students engage in a group project to deliver a product for a customer. To successfully achieve this, students are required to apply their knowledge from the Scaffolding stage, as well as to do research about the problem and any programming skills that may be relevant. Author’s addresses: Krinal Trivedi, Computer Science Department,University College London; Minh-Long Do, Computer Science Department, University College London; Jason Martyres, University College London; Yichen Lu, Computer Science Department, University College London; Ricard Gascons, University College London. ACM acknowledges that this contribution was authored or co-authored by an employee, or contractor of the national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. Permission to make digital or hard copies for personal or classroom use is granted. Copies must bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. To copy otherwise, distribute, republish, or post, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2017 ACM. XXXX-XX/2017/2-ART1 $15.00 DOI: 0000001.0000001 ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

1:2

• M. Do, R. Gascons, Y. Lu, J. Martyres, K. Trivedi

• Innovations - Third year students engage in an individual project that requires them to apply knowledge gained from both the Scaffolding and Discovery phases. The focus of this paper is going to be the ”Discovery” stage where often students are unable to get support with their projects.

2

BACKGROUND

Currently at University College London (UCL), the students go through the SDI stages. During the Discovery phase, there is currently no system in place that allows TAs to monitor each individual contribution visually. Due to the absence of a system, students currently hold meetings to agree on the distribution of one hundred points; which is then passed to the TA via email. Not only is this difficult for TAs to track progress, but they are also unable to cross check the student’s points distribution against their actual contribution.

3

PROJECT GOALS

Our ultimate goal was to create a user friendly system which relieves the hassle of team collaborations and the barrier between learning and teaching. Another goal is to make the system cross-platform and extensible so that other universities are able to deploy and modify the system on their infrastructure easily.

4

DELIVERABLES

Since this project’s aim is to create a user friendly system which could assist team collaboration and teaching activities, it will be used by both students and teacher. As a result, we designed two major components for the system.

4.1

Visual Studio Team Service Extension

This is the application used by students. It allows student to use 100-Point System in VSTS, setup connection with their project repositories for code metrics data visualization as well as Slack integration.

4.2

Teacher Dashboard

This is the application used by teachers or teaching assistants. It displays result of 100-Point distribution and code metrics in a higher level view. Teachers can use it to compare performance between teams and projects, which give teachers a easier and intuitive way to evaluate performance of different teams.

5 RESULTS 5.1 Architecture In the future, it is expected that a large number of users are going to use the system. Hence, an architecture that is fault-tolerant, scalable and loosely coupled to isolate failures within the system is required. Another element that needs to be taken into account is that the technology stack of the system should be easily upgradeable. There two types of architecture that have been considered, monolith and micro-services. The monolith is single service that serves all incoming requests whereas micro-services are split into loosely coupled services that only perform one specific business function very well [23]. When scaling out, the whole monolith has to be scaled out although only one particular business functionality is experiencing high load. However, in a micro-services system only the particular business service will need to have additional instances to cope with the increased load. Another reason for using micro-services is that deploying a service is less riskier than deploying a monolith. Because if the micro-service does not startup the system will experience a partial failure whereas in the other case it is an outage. As a consequence, choosing the Micro-service architecture is a natural choice since it has the properties mentioned above. However, the trade-off for this choice is that the developers of the distributed ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

Visual Studio Team Services: Discovery •

1:3

Fig. 1. Architecture of the System

system will have to deal with additional complexities. An example of the trade-off is that developers need to have a good understanding of the system before modification can confidently be made. Another trade-off is that the system is very complex which makes debugging very difficult, e.g. logs are distributed. Distributed log issue can be solved using log aggregation software such as Elasticsearch or Microsoft Operation Management Suite. Figure 1 illustrates the architecture of the system. In this context, there are three business functionalities, hence there are three service components. All service uses Azurefis default SSL that is provided for its its default domain fiazurewebsites.netfi [20]. Every service uses REST to communicate, they all support typical request types such as GET, POST and DELETE, refer to the REST specification in [15] for more information regarding the operations.

5.2

100-Point System

5.2.1 Front-end. The 100-point front-end consists of two parts; assigning the points and viewing the status of points distributions. In 100-point assign tab, students are able to see their team members, to whom they can allocate their distributions. A calculator is also visible indicating the number of points yet have to be allocated. Upon mutual agreement, the points distributions for the current week can be submitted, and no modifications can be made. To enable transparency and to avoid the need for lengthy conversations, students are able to view the points allocated by their team members in real time. Once the points have been assigned, students are able to use a date-picker tool to select the start of the week to which they would like to see the points allocations. Upon clicking a valid date, students are able to see the distributions through a pie chart, which is created using Chart.js and the data from a specific endpoint in the back-end through JQuery AJAX calls. A line chart is also used to display the distributions history of a particular student over time. ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

1:4

• M. Do, R. Gascons, Y. Lu, J. Martyres, K. Trivedi

5.2.2 Back-end. The service is supposed to manage the 100 point distribution of teams. It is responsible for persisting, validating and handling retrieval requests made by the VSTS front-end plug-in. When the user submits a request, the following happens. Firstly, the back-end received the 5.2.3 API. The 100-point back-end also provide end points where front-end system can pull different types of data from, such as all team member’s information, weekly points distribution, all history of points distribution and history of points distribution for a specific member. Another important job of the 100-point back-end is to validate submitted points distribution in back-end. Several rules are made in back-end validation: • Students are not able to edit the points of previous weeks, nor can they allocate points for the following week. • The number of points assigned to the team must be exactly 100. • All team members must agree on the amount of points given to each other. • All team members must be graded by everyone. • No two team members must have the same amount of points.

5.3

Code Metrics

5.3.1 Front-end. In this interface, students are able to graphically view their progress over time through the code they committed to the repository. The three graphs that they are able to see are line charts of code added, code deleted and code coverage changes. These charts are created rendered by Chart.js using the data that is retrieved from the code metrics back-end. An overall GPA is shown in text indicating the overall quality of the repository, the color of the charts changes depending on the GPA. 5.3.2 Back-end. The services focuses on being a proxy between ”Code Climate” and ”Visual Studio Team Services” for the system. This system implements a platform called Code Climate enables the automated analysis of complex code bases. Code Climate also tracks code-base metrics over time, making it easy to monitor code-base trends as well as to perform qualitative and quantitative analysis. Here is a step-by-step guide of how this system works: • The service requires a Github token to access the accounts to fetch private repositories, VSTS token to access commit statistics and Code Climate token to be able to fetch the code basefis test coverage history and GPA. These tokens are requested from the ”settingsmanagement” back-end • The contents stored in the cache are the project names from VSTS, the VSTS project IDs, Github’s project IDs and names to access Code Climate’s API. 5.3.3 API. The Code Metric back-end provided a endpoints to retrieve data commit statistics data from Visual Studio Team Services and test coverage history and code base GPA from Code Climate.

5.4

Settings Management

5.4.1 Purpose. This service is an auxiliary system that was built because integrating with third-party services through REST requires a token and is only shown and retrievable only once at the point of its creation. Hence, a service was needed that would store these sensitive information and allow applications that act as a proxy between the third-party service and the system to retrieve them on demand. 5.4.2 Front-end. The front-end of this system provides input fields for taking usersfi input of tokens such as VSTS, Slack, GitHub and Code Climate and send them to back-end service. It also displays the status of any existing tokens. 5.4.3 Back-end. The service allows retrieval and simply stores any settings and tokens that are required for accessing other systems such as VSTS, Slack, GitHub and Code Climate. ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

Visual Studio Team Services: Discovery •

1:5

Fig. 2. Technology Stack

5.4.4 Weakness. Currently, this service is a single point of failure of the system if only there is a single instance operational. To solve this issue at least two running instances behind a load balancer is required. This solution costs more computational resource but is a necessary trade-off in order to make the system fault-tolerant.

5.5

Slack Bot

Slack is a team collaboration tool that allows team members to make one-to-one or group conversations. This system purely focuses on providing the collaboration functionality when there is a significant disagreement between distributing 100 points. Slack Bot is a helper to integrate Slack with our system. An example case of using Slack could be in the event of team members disagreeing by a small margin, such transparency would create an incentive to agree on points. If there is still no agreement, students have the option to carry out a conversation on Slack to discuss the distributions further. 5.5.1 Back-end. The Slackbot main functionality is to act as a proxy between the 100-point back-end and Slack groupfis channel based on the configuration of the user.

5.6

Technology Stack

Below is a list and description of all the technologies used in this project: • Visual Studio Team Service Web Extension Software Development Kit (VSS SDK) is the core client SDK script for developing Visual Studio Team Services extensions. It enables web extensions to communicate with Visual Studio Team Service.[25] • Azure is for for building, deploying, and managing applications and services through a global network of data centers managed by Microsoft. • GitHub is a web-based Git or version control repository and Internet hosting service. It is designed to manage small and large codebases in a distributed fashion and also provides a lot of integration with other services, such as Travis CI. • Travis CI is a CI platform that executes commands based on the .travis.yml file when a code is pushed or merged. We have followed the Continuous Integration methodology to detect any integration issues ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

1:6

• M. Do, R. Gascons, Y. Lu, J. Martyres, K. Trivedi

• •

•

• •

• • •

5.7

early. We had initially planned to construct our CI infrastructure using Kubernetes, Jenkins and Azure Container Registry. The cost to maintain this service was around $1000 per month. Such a sum was clearly out of the budget of the project, hence an alternative was Travis CI that was free [7]. Django is a web framework to create web services. This framework has been used as a base for our RESTful web services and Web Appv[10]. Django REST framework is an extension for Django for building Web API’s. It supports Web browsable API which is very useful for developer documentation for APIs. This framework is used to create RESTful back-end Web services in this project [6]. PostgreSQL is the most advanced open source database [21] that is well supported by Django. This was chosen over NoSQL because nature of the system requires to have ACID properties which a SQL would be able to provide. A NoSQL usual would provide record-level atomicity but does not guarantee that a series of transaction will succeed [24]. It is essentially a scaling and transaction management trade-off. Redis is an in-memory data structure store that can be used as a database, cache and message broker [18]. Docker is a container that allows developer to manage the dependencies of their application manually through a Dockerfile instead of relying the host machine to have them installed [11]. It can be seen as a form of a lightweight virtual machine built for developers. We use Docker to separate application processes from each other in a container so that they do not affect each other if anything goes wrong. JQuery is a well known JavaScript library [16]. We have used it for all kinds of front-end tasks such as manipulating DOM, firing AJAX calls, parsing JSON objects and front-end validations. Bootstrap is a CSS and JavaScript framework for developing responsive websites [3]. It has been used to develop the layout of the interface of the VSTS plug-in. Chart.js is an open source JavaScript library which supporting eight different chart types and is responsive. It renders charts using HTML5 Canvas which has a great performance across all modern browsers[5]. Chart.js is used to visualize data in the VSTS extension and Teacher Dashboard.

Deployment

Each servicefis code base has a Dockerfile that contains a sequence of instructions that is read by Docker to build a Docker image [12]. Each repository, in this case GitHub repository, is connected with Travis CI and contains a .travis.yml file which instructs Travis CI what to do for new changes. In our case, we instructs Travis CI to run all of our unit tests, if the change has been merged to the master branch then build a Docker image which is then pushed to Azure[13] container register. For each successful merge into a master a Docker image is built, tagged and pushed to Azure container registry, The tagging of the Docker images follows the semantic versioning convention, e.g. x.y.z where x, y and z are the major, minor and patch versions respectively, refer to [22] for more information. Finally the Docker image is stored and is deployed through Azure’s Web App on Linux [1]. Web App on Linux automatically fetches the Docker image based on the given repository, image name and tag.

6 SOFTWARE ENGINEERING PRACTICES 6.1 Continuous Integration Continuous Integration (CI) is a practice that requires developer to integrate code into the main branch several times a day. Each commit is checked against the master branch for conflicts and run against a set of tests, possibly ranging from unit to behavior driven tests. CI is done to locate errors early on before it gets into the ”master” branch. When errors are located early, it is easy to isolate them and therefore it is faster to fix them [26]. For the current project Travis CI was used because the team has prior knowledge about it. However, it is possible to use other CI platforms such as Codeship [9] or CircleCI [8]. Travis CI has been configured to the GitHub ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

Visual Studio Team Services: Discovery •

1:7

repository for this project. Whenever, new commits are pushed to this repository, or there is a pull request, GitHub will notify Travis CI of this. Travis CI checks out the updated branch and run the commands and build the environment specified according to the .travis.yml file. Provided the build is successful; meaning the specified commands have exited successfully, e.g. running unit tests and compiling source code. Travis CI will proceed to create a Docker image based on the Dockerfile if it is currently compiling the masterfis branch code base. Each master’s branch build follows the following semantic versioning ”{major version}.{minor version}.{build number}”, in this context it is 1.0.{build number}. Finally, Travis CI will push this Docker image to the registry on Azure.

6.2

Continuous Delivery

Each Docker image has the following structure: base-URL (which is the name of the repository), an image name and an image tag. These three pieces of information is needed to pull a Docker image from a private registry. The only step to deploy a new version of the service that is dockerized is to manually to change the version number in the application’s setting of Web App on Linux on Azure.

6.3

Testing

Prior to implementing JavaScript functions for the front-end, we created created unit tests to ensure the functions behave correctly. The testing framework that had been chosen is Jasmine since it supports Behavior Driven Development (BDD), in addition to its ease of use. It is recommended to replace calls to a database with mocks [4]. Following this practice, fifty-six test cases were created to ensure the correct behavior of the functions. On the other hand, the back-end did also write test cases following Django Test Framework. The code coverage of the front-end systems is around 60% on average.

6.4

Project Management

Throughout this project, we followed the Agile methodology. The greatest advantage for using Agile in this project is flexibility. Customers are able to get frequent progress updates and are able to change requirements if needed. By meeting the stakeholders of our project on a weekly basis to review our progress, we were able to get prompt feedback. This allowed us to re-prioritize any tasks if needed. A variant of agile was used in this project known as scrum. Our development was split into weekly iterations known as sprints, where the aim was to incrementally increase the functionality of the software. Front-end and back-end development worked in sync to first deliver on the ”100-point” system, followed by the ”code metrics” system and finally on the ”teacher interface” system. Regular meetings were held to assess progress, reassess priorities and to resolve any issues affecting progress. The three systems mentioned were the epics of this project. Each of these epics were divided into user stories. To maintain the product backlog and sprint backlog, we made use of the Issues feature on GitHub. All tasks were initially written by the scrum master, and a subset of tasks were assigned to a team member every sprint. Upon completion of a task, the team member would make this task a closed issue. If a bug was found, a team member was able to create another issue. Depending on the urgency of this issue, the tasks assigned to the team member may need to be re-prioritized.

7

LESSONS LEARNT

During the project we have experienced that the technology chosen at the beginning that believed to be appropriate in theory do not necessarily work in practice. Hence, the design of the code base of the application needs to be modular in order to cope with a lot of moving parts during the development iterations which our code base was designed to be from the beginning. A real example of this would be that our the caching mechanism for the back-end was Memcached. However, Microsoftfis recommendation was to use Redis instead, therefore ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

1:8

• M. Do, R. Gascons, Y. Lu, J. Martyres, K. Trivedi

Memcached was replaced with Redis [19]. In order to make this a flexible application, environment variables needs to be used as flags.

8

CONCLUSIONS

With the help of VSTS, the monitoring of individuals who are taking part in the Discovery stage can be made more easy and accurate. By developing an extension of VSTS, we have made it easier to graphically view the history of the one hundred points distributions. Assigning points for the current week is also simpler, as team members simply enter their distributions in the extension. By being able to view the distributions that have already been submitted by other team members for the current week, students should be able to decide on the distributions more easily. If this is not the case, then students are able to conduct a meeting remotely through Slack. Storing and retrieving data from the back-end happens via JQuery AJAX calls to specific endpoints. In order to verify a student’s contribution with their actual output, the extension also performs metrics on the student’s committed code. A platform called Code Climate is implemented which performs analysis of complex code bases and tracks code-base metrics over time. Through the output of a GPA, a student’s contribution can be verified with their output.

9 FUTURE WORK 9.1 Fault Isolation Our current system is made of micro-services, however to make the system more fault-tolerant it is best to create a separate service for each business functionality. By doing this, it will increase the system’s complexity and make it difficult to maintain because developers have to understand the system before any modification can confidently be made.

9.2

System Integration

Currently, the web application ”Teacher Dashboard” uses its own PostgreSQL database to authenticate user. Using this method it will require the users to maintain two difference accounts. To improve this connecting ”Teacher Dashboard” with the organisation’s LDAP will decrease account maintenance issues for the user. Integrating with LDAP will allow users to use their existing accounts.

9.3

System Optimization

There are a number of performance improvement that could be made but was not done due to time constraint. Firstly, for the 100-point back-end caching is needed to reduce the load to the database. With low number of users using the system, loading the database is acceptable. However, taking into consideration that in the future a large number of users are going to use the system, the database will not be able to handle the load if every single query requires the application to retrieve data from the database. Secondly, after implementing the cache, designing the sharding mechanism on the request and database level [2]. If sharding is done correctly then all application instances will have in theory the same load and the content of the application caches will be unique.

9.4

Application Log Management

When a system contains a lot of services and has become large, it is necessary to be able to monitor them. One of the ways of doing this is to aggregate logs of the services into a centralized location with tools such as Elasticsearch [14] or Microsoft Operations Management Suite [17]. Once the logs are aggregated, it is possible to use them for analysis, such as how often a feature is used through the number of times a request is sent to a particular endpoint. ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.

Visual Studio Team Services: Discovery •

1:9

REFERENCES [1] ahmedelnably & Rebecca Agiewich & Wesley McSwain & ShawnJackson & Cephas Lin & Kim Whitlatch & Tyson Nevil & Apurva Joshi & Glenn Gailey. 2017. Create an Azure web app running on Linux. (2017). https://docs.microsoft.com/en-us/azure/app-service-web/ app-service-linux-how-to-create-a-web-app [2] Masashi Narumoto & Christopher Bennage. 2017. Sharding. (2017). https://docs.microsoft.com/en-us/azure/architecture/patterns/ sharding [3] Bootstrap. 2017. Bootstrap. (2017). http://getbootstrap.com/ [4] Tim Buschtns. 2014. How to write better Jasmine tests with mocks. (2014). http://eclipsesource.com/blogs/2014/03/27/ mocks-in-jasmine-tests/ [5] ChartJS. 2017. Chart.js Documentation. (2017). http://www.chartjs.org/docs/ [6] Tom Christie. 2017. Django REST Framework. (2017). http://www.django-rest-framework.org/ [7] Travis CI. 2017. Travis CI Documentation. (2017). https://docs.travis-ci.com/ [8] CircleCI. 2017. CircleCI. (2017). https://circleci.com/ [9] Codeship. 2017. Codeship. (2017). https://codeship.com/ [10] Django. 2017. Django Project. (2017). https://www.djangoproject.com/ [11] Docker. 2017. Docker. (2017). https://www.docker.com/ [12] Docker. 2017. Dockerfile reference. (2017). https://docs.docker.com/engine/reference/builder/ [13] Azure documentation. 2017. Azure documentation. (2017). https://docs.microsoft.com/en-us/azure/ [14] Elastic. 2017. Elastic. (2017). https://www.elastic.co/ [15] Sandro Hawke. 2017. REST. (2017). https://www.w3.org/2001/sw/wiki/REST [16] jQuery. 2017. jQuery API. (2017). https://api.jquery.com/ [17] Microsoft. 2017. Operations Management Suite. (2017). https://www.microsoft.com/en-gb/cloud-platform/operations-management-suite [18] Microsoft. 2017. Redis. (2017). https://redis.io/ [19] Microsoft. 2017. Run your Memcache app with Azure Managed Cache Service. (2017). https://msdn.microsoft.com/en-us/library/azure/ dn386122.aspx [20] Microsoft. 2017. Secure your app’s custom domain with HTTPS. (2017). https://docs.microsoft.com/en-us/azure/app-service-web/ web-sites-configure-ssl-certificate [21] PostgreSQL. 2017. PostgreSQL. (2017). https://www.postgresql.org/ [22] Tom Preston-Werner. 2017. Semantic Versioning 2.0.0. (2017). http://semver.org/ [23] Chris Ricardson. 2017. Microservice Architecture. (2017). http://microservices.io/ [24] Martin Samson. 2017. Why NoSQL is better at fiscaling outfi than RDBMS? (2017). http://stackoverflow.com/questions/8729779/ why-nosql-is-better-at-scaling-out-than-rdbms [25] VSTS Web Extension SDK. 2017. VSTS Web Extension SDK. (2017). https://github.com/Microsoft/vss-web-extension-sdk [26] Throughworks. 2017. Continuous Integration. (2017). https://www.thoughtworks.com/continuous-integration

Received February 2007; revised March 2009; accepted TBC

ACM J. Comput. Cult. Herit., Vol. 9, No. 4, Article 1. Publication date: February 2017.