international group report - study with me
TRANSCRIPT
International Group Project
Study With Me
Group Project Report
Contents
Introduction...........................................................................................................................................6
About this Document.........................................................................................................................6
The Team...........................................................................................................................................6
Management of the Project..................................................................................................................7
Foundations.......................................................................................................................................7
Team Structure..................................................................................................................................7
Software Development Methodology...............................................................................................8
Schedule Structure............................................................................................................................9
Issues...........................................................................................................................................10
Solution.......................................................................................................................................11
Team Communication.....................................................................................................................12
WhatsApp....................................................................................................................................12
Skype...........................................................................................................................................13
Email............................................................................................................................................14
Google Drive & GitLab.................................................................................................................14
Trello............................................................................................................................................15
Initial Design Overview....................................................................................................................16
The Client.........................................................................................................................................16
Specifications...................................................................................................................................17
Risk Analysis....................................................................................................................................17
System.............................................................................................................................................17
User Interface..................................................................................................................................18
Validation............................................................................................................................................21
Platforms – Android Application......................................................................................................23
Implementation...................................................................................................................................27
Programming Languages / DB analysis............................................................................................27
DBMS/ERD Design...........................................................................................................................27
Implementation of DB.....................................................................................................................27
Server Development, testing and Environment...............................................................................28
System architectures...................................................................................................................28
Development platform................................................................................................................29
Deployment specifics configuration................................................................................................30
Https............................................................................................................................................30
Mail..............................................................................................................................................30
Backup.............................................................................................................................................31
Integrated Workflow.......................................................................................................................32
Application platform........................................................................................................................32
Webserver...................................................................................................................................32
Https............................................................................................................................................32
Caching........................................................................................................................................33
Database..........................................................................................................................................33
Mysql...........................................................................................................................................33
Phpmyadmin................................................................................................................................34
Security features..............................................................................................................................34
Https............................................................................................................................................34
Ssl configuration..........................................................................................................................34
fail2ban........................................................................................................................................35
Pemkey........................................................................................................................................35
2 factor authentication................................................................................................................35
Patching...........................................................................................................................................38
From development infrastructure to production/demonstration...................................................38
Requirements..............................................................................................................................38
Needs...........................................................................................................................................39
Overview of the changes.............................................................................................................39
Architecture.....................................................................................................................................40
Domain Name service..................................................................................................................41
Mail sending................................................................................................................................41
Database......................................................................................................................................42
Testing.........................................................................................................................................42
Production webserver.................................................................................................................42
Ami (Amazon Machine Image).....................................................................................................42
Automatic scaling group..............................................................................................................42
Scaling policy...............................................................................................................................43
Elastic Load Balancer.......................................................................................................................43
Principe........................................................................................................................................43
Implementation...........................................................................................................................43
Issue:............................................................................................................................................43
Proxy/caching layer.........................................................................................................................44
Principe........................................................................................................................................44
Overview of the flows......................................................................................................................45
Testing.............................................................................................................................................45
Methodology...............................................................................................................................45
Results.........................................................................................................................................46
Interpretation..............................................................................................................................46
Data analytics:.............................................................................................................................46
Way forward....................................................................................................................................46
Pricing..............................................................................................................................................47
Language Framework Introduction..................................................................................................47
PHP......................................................................................................................................................47
Advantages......................................................................................................................................47
Disadvantages..................................................................................................................................48
PHP Development Approach...........................................................................................................48
PHP Prepend....................................................................................................................................48
Student Class...................................................................................................................................49
PHP Slim API....................................................................................................................................50
PHP Slim Documentation.................................................................................................................51
PHP Coding Practice.........................................................................................................................52
PHP Security....................................................................................................................................52
File Structure...................................................................................................................................54
References...........................................................................................................................................55
Notable Features.................................................................................................................................55
Notifications....................................................................................................................................55
Rating System..................................................................................................................................56
Android implementation.....................................................................................................................57
Notable Feature - Widget................................................................................................................57
Background Management System...................................................................................................59
Testing.................................................................................................................................................59
PHP Function Tester....................................................................................................................60
Risk Evaluation/Major Issues...............................................................................................................61
Evaluation of the Product....................................................................................................................69
Project Evaluation................................................................................................................................78
Conclusion...........................................................................................................................................79
Thanks Sections...................................................................................................................................79
Appendix 1.0........................................................................................................................................80
Appendix 1.1....................................................................................................................................81
Appendix 1.2....................................................................................................................................81
Appendix 1.3....................................................................................................................................82
Appendix 1.4....................................................................................................................................87
Appendix 2.0........................................................................................................................................89
Appendix 2.1....................................................................................................................................89
Appendix 3.0........................................................................................................................................90
Appendix 4.0........................................................................................................................................94
Appendix 5.0........................................................................................................................................96
Appendix 5.1....................................................................................................................................97
Appendix 6.0........................................................................................................................................98
Appendix 7.0......................................................................................................................................100
Appendix 8.0 – Complete Minutes....................................................................................................100
Appendix 9.0 – Communication Breakdown Emails..........................................................................114
Appendix 10 – Sample Code..............................................................................................................116
Appendix 11 – Web User Manual......................................................................................................121
Appendix 12 – Android User Manual.................................................................................................137
Table of Figures.....................................................................................................................................8
IntroductionAbout this Document
KR For such a large project, it should come to no surprise that all of the information was not retained by one individual member but in fact, each member had their own area of which they were most knowledgeable in. Due to this, all of the Scottish members contributed to the creation of this document to provide accurate and concise information. To differentiate between authors, the reader should be aware of the initials placed as superscript at the beginning of chapters, paragraphs or sentences and can assume that the author will remain until a new occurrence of initials.
Each set of initials relates to the following key:
This report will include detailed results of the techniques used whilst progressing through the production of the product and the following development processes: analysis, design, implementation and testing. It will also identify the authors of each obtained piece of work that was proven viable to the overall creation of the application and hence, outline the acquired tasks of the team members. Towards the end of this report there will be an overall evaluation of the product and the process endured during the production; this will cover what worked well and what would have been done differently but first, a brief insight into the structure of the team to ensure a better understanding of the inner workings of the project.
The Team
The individuals contributing to the creation of this product were part of fourteen person team originally, but this was reduced to thirteen when one member decided to leave due to their personal schedule. Collectively, there was a large scope of skills and qualifications associated with the team allowing versatility in reference to the techniques used. Six of these members were students associated with Edinburgh Napier University and the other seven students were from Zhengzhou University of Light Industry.
KR Kim Reid 40170312
CB Calum Beck 40099589
PK Patrick Kelly 40100046
AN Antoine Noel 40174355
MA Max Atkinson 40089245
NH Nicholas Hunt 40092121
Management of the Project
Foundations
Prior to official beginnings of the project, an abstract schedule had already been created by the leading supervisors of the intended project in the form of three stages. These stages consisted of:
Stage One:
The students from Zhengzhou would visit Scotland for two weeks where the foundations of the group would be built and the initial plans for the project would be discussed.
Stage Two:
All students would reside in their own countries whilst continuing development on the project through their respective and preferred methods of communication.
Stage Three:
The students from Scotland would visit Zhengzhou for two weeks to conclude development, test the application and finally, present the project to an audience of imperative individuals within the academic community (Zhengzhou).
The project managers would consider the above stages as boundaries and important milestones within the project that the schedule would revolve around. The process of creating the initial schedule and the structuring of the team was as follows.
Team Structure From a basic project management point of view, several questions had to be considered when creating the first structure of the group, a small selection of these would be:
- How many specifications are there?
- What are the required skills set of each of these specifications?
- Which members would fit the roles required to implement these specifications?
The answers to questions similar to the selection above would give the best foundation to build the schedule from; where all of the team would work in unison on one particular area of specifications during each phase.
With such a large group it was important that the roles within the group were clear and each member understood their point of call; which person could deal with which type of issue. As a result of this thought, it was decided that each area would have an individual who would make decisions and deal with issues relevant to their skills and knowledge, for example: a head developer. Each role given to a Scottish student was mirrored by a Chinese student; this was suggested by the lecturers
present at the beginning of the project to promote genuine team work between the two groups of students.
The role selection was an easy process that consisted of: defining the required skill area of each role needed to complete the project, taking into consideration the particular roles preferred by each individual member and most imperatively, the skill set of these members. A visual representation of this structure can be found at Appendix 1.0.
Software Development Methodology
To implement this, the Agile technique was chosen after research via different source types such as: a presentation on Project Management from Brian Davison and from reading articles like, ‘A Guide to Selecting Software Development Methodology’ (Faridani, 2011) which explains in detail each of the available software development methodologies. Agile is a general term that refers to any development process that follows the rules of the Agile Manifesto and is now commonly used within industry due to its’ positive success rates. A basic definition of the methodology would be that, the proposed system is split via the requirements into separate phases and dedicated a specific time period to be completed. The implementation within the phase is tested or development moves onto the next face dependent on bugs, issues and the whether the client is satisfied. This process can in theory, never end as a system can constantly require more functionality or fixes. Advantages to using an Agile development methodology that are relevant to the project are as follows:
- Test Driven Development – This is beneficial to a system of high complexity as it ensures functions aren’t being built upon an already fault foundation; it encourages a more stable application and decreases the risk of bugs being unnoticed.
- Flexibility - Agile promotes change and hence, is not restrictive when considering specifications allowing them to change as the system is implemented and shaped.
- Enjoyable team dynamic – Traditional software development methodologies rely on intense documentation for defining specifications and continuous progress reports during the project. The Agile methodology endorses team discussions, daily meetings and instead of long duration project plans, short term milestones that fit the specific project in question are created. In theory, this empowers the team as large quantities of the decisions are made by the whole team rather than a management figure within the group; promoting a better work environment.
With this acquired knowledge, a calculated decision could be made and Agile seemed to be the best fit as it could support the idea of iterative development where multiple versions of the application would be produced, each more functional than the last. In basic terms, it was made clear by the supervisors that there must be a working version available to present in China and so, Agile allowed the team to implement a basic working version within the first stages of the project but then have the chance to experiment with idealistic functions with the remaining time.
Traditional Agile would state that phases should be restricted within specific periods of time known as time boxes to reduce the risk of the never ending development but it was decided that this wouldn’t work well for this project due to it being the first of its’ kind. Therefore, the plan was to use a slightly less restrictive version of Agile, allowing for the time boxes to be flexible; as it was obvious that a variety of unknown territories such as: the international communications, could cause delays that a normal project would not necessary face.
Schedule Structure
During stage one of the project, both project managers discussed a schedule that would be produced relative to the information researched and stated above about the methodology, Agile. Phase one would be the classed as the schedule created by the lecturer, Brian Davidson prior to the Chinese students arriving in Scotland and consisted of the common analysis stage of a project. Phase two would be created by the project managers and defined by the most imperative requirements of which will be detailed at a further point within this documentation. This seemed like the most logical way of guaranteeing a functional product to present in China, even if it was on a basic level and then allowing the more idealistic functions to be implemented within phase three. To fairly split the tasks and truly work as a team, it was initially decided that each of these tasks would be allocated to at least one member from both sides of the team to work on together. The schedule would also incorporate tasks being completed simultaneously as the team was of such a size that tasks not dependent on other tasks could be completed at the same time; increasing the velocity of progress. The separation of tasks and progress of the project would be defined as milestones within the schedule; these would also act as specific targets for the team members to reach within certain time periods. Logically, the main milestones would mirror the common areas of development found within the Agile methodology template: planning, design, implementation and testing. Milestones are a good source of gaging the expected duration of a phase, it is easier to set a restrictive time limit between milestones but allow flexibility within that duration (time taken for tasks).
For such a large project both specifications and member wise, it was more efficient to store this schedule within a visual representation and an environment that would allow for changes to occur with ease. There are many software applications that provide such an environment but it was more suitable to use Microsoft Project 2010 due to the specialist skills set of the Scottish project manager and their experience using this software. This application allows for the tasks to be input along with an estimated duration time, the overall start/end date of the project and the hours in which the team members would work daily. The next step would be to allocate members to each task and then based on all of the previously stated information including the amount of members allocated to the task the application will automatically schedule the project. Different views of this schedule are available such as: resource sheet (the members), task sheet (the tasks and dates) and of course, the most important to this project, the Gant Chart view. Below is the screenshot of the initial schedule and the resource sheet can be found at Appendix 1.1.
In addition to the above schedule, the development team also had a list of the PHP functions that had to be written alongside a description of what each on did and the data from the database they would acquire. The developers would then be able to choose from the functions still needing to be done and track their progress but also, refer to the descriptions of the other functions in case they could be reused.
Also, during the third stage of this project a separate schedule was produced for the two weeks that the Scottish students visited China; this can be found at Appendix 1.4.
Issues
The initial schedule did not take into consideration some circumstances that would occur during the first weeks of stage two and hence, it did not portray a realistic representation of the project. These issues were as follows:
- The students from Zhengzhou would not immediately start the project when arriving back home in China as they were on an academic holiday to celebrate the Chinese New Year; this was fine but unexpected.
- Both sides of the team were at different parts of their academic year: the Scottish students were at their busiest part of the year with exams and coursework deadlines, whereas, the Chinese students were merely starting their term.
- Another fact that separated the circumstances faced by the two different cultures was part-time jobs: all of the Scottish students worked part-time alongside being a fulltime student which is normal in the British culture but not so common in the Chinese culture.
Alone, any of these issues wouldn’t have had much of an impact on the schedule and overall progress of the project but together, they most definitely caused delays. The first issue stated was easily resolved by updating the Gant Chart and merely eating into the reserved time at the end of the project for niceties to be implemented; this updated Gant Chart can be found at Appendix 1.2. The second issue and third issue could only be resolved by the Scottish students making up for lost project time during the two weeks during the “Easter Holiday” which are usually reserved for studying but this definitely had a positive effect on the momentum of the project. Differences in culture resulted in the Chinese students unable to empathise with the fact that the Scottish students would produce work in bulk rather than a steady flow throughout stage two; due to their busy schedules.
Solution
It was clear that the current structuring of the project with members from both teams working simultaneously on tasks was not working as the issues noted above resulted in situations where the Chinese students would be waiting on tasks to be completed by the Scottish students before they could continue on with their work. Based on this, it was decided that it would be more efficient for the Napier students to work mainly on the website development and the Zhengzhou students work mainly on the android application; this also was a better when considering skill set. Even though each side of the team had there dedicated platform to focus on, at the weekly meetings, both platforms were discussed and issues were worked on together as a group. Also, if skill sets required were available from the any of the other members then it was encouraged that they were queried and helped with any related issues. In terms of management, it was assumed that with this change, the project managers would create the schedules associated with their own particular side of the group but still ensure that the corresponding project manager was aware of what was going on. Overall, this change in the development method was risky but paid off in the form of increased progress.
As the end of stage two approached, it was clear that even with the above solutions in place, the progress of the project was not satisfactory and so, a more intense ‘week by week’ schedule was created alongside a ‘progress so far’ documentation for the opposite side of the team to view. It was important that the functions were ready by stage three so that they could be brought together and implemented in both platforms of the application which was the thought behind the creation of the new scheduling documentation. The Scottish scheduling documentation is located within Appendix 1.3.
Team Communication
It was important that during all stages of the project that the communication was concise and constant to ensure that a solid understanding was common amongst all members of the team. If this understanding was not present even within one individual member then this increased the risk of the project losing momentum. Many of the scheduled tasks were dependent on the completion of another task, so it was important that each member worked in unison where necessary; one person lacking could affect the work of others.
Whilst all of the students were in the same country the main form of communication was face to face conversations but this was not applicable during stage two where all students were in their residing countries. Even when using internet technologies to communicate with each other, issues such as time difference had to be taken into consideration; at the beginning of the project there was an eight hour time difference but this decreased to seven half way through stage two. Dealing with such circumstances was new to many of the members within the team and could sometimes cause a hindrance in the form of frustration when waiting for replies. A protocol was quickly formed during this stage that involved several different platforms for communication of which follow:
This is cross platform messaging application for a variety of different mobile phone applications and hence was used for daily communications. Although the messages to the Chinese students would be received and replied to as soon as possible, this form of communication was much better in real time; where the conversation could flow continuously for a respectable period of time. Even though the Chinese students’ English skills were impressive, it couldn’t be denied that they were better at interpreting written English through with the added support of translation applications and therefore, using communication channels such as WhatsApp was incredibly beneficial when explaining complex and technical terms. This application also supplied effective group chat functions and the ability to send media (pictures, audio clips etc.) efficiently, which was again, beneficial to the communication between the students.
There were two groups on WhatsApp: one merely for the Scottish students that was used for conversing about everyday matters, progress reports, questions and general team bonding. This would also be used for arranging the details of non-formal meetings: time and place etc. The other group on WhatsApp was the international chat, where all members from both Scotland and China were present. This was used mostly during our formal meetings (discussed later) as a back up to ensure that each individual understood what was being said or even if the internet connection was bad causing the audio between groups to be difficult to comprehend. It was also used for a more casual conversation between the members; it was important that the team bonded well and felt that each member was approachable; this was a good environment to promote such a dynamic. As many messaging applications now offer, WhatsApp allowed for everyone in a conversation to see when a message had been read and who by. This functionality stopped any messages being purposefully ignored and confirmed that an individual had in fact received a message which could later be referred back to, if need be.
WhatsApp was not known amongst the Chinese students but due to its’ simple layout and functionality, they were able to use it comfortably within a short period of time. The Appendix 2.0 shows an example of the WhatsApp environment used during this project; the group chat titled ‘The
Great Firewall’ is available to only the Scottish students and hence, the chat titled ‘International Group Project’ is public to all of the group members.
Issue
The only issue that occurred whilst using this form of communication was the fact that when everyone was present within the conversation it could get cluttered and confusing with the possibility of different conversations taking place within the one group chat. This could cause important points defined within the conversation to be missed and on a social level, individuals could feel like their opinion was not valued.
Solution
One solution for this issue was simply to dedicate one individual to write within the conversation when possible i.e. when groups of the members are together and another solution was for each individual to only send messages relevant to the current subject.
Skype
Sometimes it is hard to get a true reading on whether an individual understands something when using written forms of communication hence, it was important to have video conferencing as part of the communication protocol. These meetings via Skype (a widely used video and audio conferencing application) were of a formal structure including the use of agendas and minutes. This type of meeting was always intended to occur weekly and be used for an overall update of progress, questions from either of the sides and of course allocating the tasks for the following week. After much discussion and compromise from both sides of the team, it was agreed that with time difference taken into consideration, these meetings would take place every Tuesday at 11am for the Scottish students and 7pm for the Chinese students. In an idealistic world, all members would be present at all occurrences of these meetings but realistically, this was never going to happen; it was just made clear to the members how important attendance would be and that they should attempt to always be there.
As stated above, these meetings were of a formal structure and hence, required extra documentation to be created for each one; agendas and minutes. The agendas were made prior to the meeting by either of the project managers and defined the main points that had to be discussed during the Skype call. This efficiently ensured that the limited time was used appropriately by keeping the meeting from going off on a tangent. This type of meeting was used to make decisions and plan progress for the next week therefore, it was imperative that everything that happened during the meeting was recorded in written form known as minutes. The role of minute taker alternated through the Scottish students with every meeting that occurred and this individual would be required to write up the notes in a standard format then place them in a shared drive following each session. These minutes were incredibly useful for several things such as the following:
- Referring back to decisions made.- Recording task allocation - Detail deadlines for these tasks - Define the details of the meeting for those who didn’t attend - Reference to who did certain tasks- Recording attendance
- Allowing the supervisors to be up to date with the process and progress of the project
The latter of which is the most imperative. Although, the Scottish team supervisor, Robert Ludwiniak was present within most of these meetings and regularly met with the members in Napier for an overview of progress, he was still able to read the minutes for a better understanding of the inner workings.
Issues
This method of communication relies on a strong internet connection which is definitely a rarity in China and so it wasn’t often that the video display was clear enough to truly be beneficial. Also, delays in audio could easily cause confusion when the language barrier was already a difficulty.
Solutions
When the internet connection wasn’t strong enough for a good quality video call then it would become a voice call and if this still wasn’t good enough for a productive meeting then WhatsApp would be used. There was no way of knowing when this issue would occur but having this protocol in place meant that it never stopped a meeting from taking place.
Email was used when the message being sent was important or had to be seen by specific members of the group. As a more formal tone of communication, it was mainly used between the project managers on a weekly basis and for communicating with the supervisors when need be. It was easy to refer back to these emails with use of the search facilities present within most of the email applications and this was made easier when the subject was titled correctly.
Google Drive & GitLab
The previous methods mentioned have all been about social communication but it was important that the team communicated well on another level; in other words, file sharing. There were two storage areas each dedicated to different types of files and accessible by all of the team members.
The Google Drive storage area was specifically used for storing the documentation aspect of the project such as: Gant charts, requirement and meeting related documents. It was highly organised within appropriately named folders once the volume of files became too much for a single folder structure. A screenshot of this storage area can be found at Appendix 2.1.
GitLab was the second storage area dedicated to storing all versions of the system both related to the android and website platforms. Each member within the development team could ‘push’ their code to the Git server for others to access and work on when need be; increasing productivity. This protocol of storing the system files became useful when issues occurred within the code that could not be fixed as it could be simply rolled back on GitLab to a working version.
Issue
China is restricted from using anything Google related applications and hence, without use of VPNs it would not be accessible via the Chinese servers.
Solution
At least one member from each side of the team had a VPN to allow access to the drive when visiting or resident in China and any important files needed for stage three were downloaded and stored locally on hard drives to ensure they were available when needed.
Trello
As stated previously within the document, the chosen methodology was Agile meaning that each phase had an iteration of: analysis, design, implementation and testing. Keeping track of the tasks completed within this cycle and the individual who undertook them could prove challenging in such a big group/project hence the use of Trello.
Trello is an online application that provides an interactive representation of scheduling and can be accessed by all of the group members once they have been accepted via the admin users which in this case, were the project managers. There are many useful functions available within this application such as:
- Creating cards- Creating tasks - Moving tasks to other cards with ease- Attaching files to tasks - Commenting within tasks- Allocating members to tasks
With the ability to use the above functions within a simplistic environment and the visual representation of the Agile development cycle proved incredibly effective within this project. The screenshot below shows the Trello environment and that the cards were titled appropriately.
Initial Design Overview
The Client CB
For our final project, our main client was those brought to us by the ZZULI students in the form of the student body from the Chinese campus. The problem presented arose from the amount of students studying in the university campus and the level of difficulty to find space in which to work collaboratively. Within ZZULI, there is more than 15,000 students enrolled1 resulting in a barrier in which to meet likeminded students to work on projects or study for established classes.
The students in China also have a distinctly different time tables from students in the UK. Instead of having 3 modules covering wide topic ranges, their system culminated in up to 10 compulsory, specific topics in which they must attend each week. This resulted in timetables which extend from 8am until typically 6pm leaving little time to discover spaces in which to work, or socialise with the student body at large in order to mix talents and build projects on a larger scope.
In the ideal end product, the students wouldn’t just be limited to the current campus, but have access to all the universities assets. We were informed that a new campus, with new furbishing and further technologies, had been built and was accessible by the students. It was hence our job to create a system which allowed the users to find other students and book spaces in which to meet and interact with one another. The specific reasons behind groups varied, so a versatile system which allowed for general group assembly was required. For more specific tasks such as group study for classes in between timetabled classes, room booking, meetings, and a communication feature would need to be implemented.
As this was an academic project to promote international connections between universities in Scotland and China, it was noted that students within the development team empathised with the problem at hand and agreed that the system was befitting of Edinburgh Napier as well. As it currently stands, group facilities exist in the Napier campuses in order to help the students but are not readily available to check online. It was noted that everything had to be arranged in person and on further inspection when talking with lecturers, the current system for teachers proved cumbersome.
Extending the system to work over multiple Universities allowed the team to use their own experiences as students currently studying within the university system to produce a system based on daily needs relative to the target audience. As previously stated, this was an academic venture, so in order to ensure a working system at the end, incorporating the Merchiston campus into the system would allow us to reduce the risks of reliance on the ZZULI data.
From this, a basic set of Design Requirements were incepted; the system was basic primarily for students so had to be quick to use, easily accessible and, as It would be used on the go as well, low on data consumption. Another point to note is we were informed that the internet speeds on the campus and in China in general, were much slower than those available here. This meant we had to take into account the amount of page loading and data retrieval.
1 http://en.zzuli.edu.cn/s/124/t/444/e2/da/info58074.htm
Our last limitation was that of Time. Although the Scottish team’s grade relied on the report and presentation of the work completed, the Chinese side required a finished, working product in order to pass. This meant that the team would have to ensure functionality of the system.
Specifications
The specifications we identified based on the specifics of the client needs. This application would have many main points of intended functionality such as: the ability to book available seats within free rooms, create study groups specific to subjects etc. and of course, join these groups.
There were many specifications discovered and agreed upon during the primary stages of this project which can be found within the Specifications Document – Appendix 3.0. These specifications were prioritised using the MOSCOW technique in the early stages of the project with the intention for all of the stated ‘Must Haves’ to be completed before the deadline and for the rest to be treated as niceties rather than necessities. As shown in the document found at Appendix 1, these specifications vary from the technical aspects to the user experience (user interface etc.) and hence, to fully complete the system, a variety of skills sets were needed collectively as a group.
Risk AnalysisKR This part of the document refers to the initial risk analysis where the original risks were defined at the start of the project; this can be found at Appendix 4.0. More risks became apparent whilst working through stage two and precautions were quickly put into place to either neutralise the possible resulting effects on the project. A small amount of the defined risks evolved into real issues but due to the planning within stage one and a quick initiative from the team members, they were dealt with before resulting in the failure of the project. These problems can be found within the Risk Evaluation section of this document.
SystemCB Based on the specifications outlined, the basic summary for the system was as such:
- Centralised Database to store all needed information (this can be seen in further detail in the implementation portion under the Database Section)
- Website portal; built to work on handheld devices as well (in the case of iPhones)- Android companion application for more succinct handheld experience
As such, the android application and the website would use the same PHP functions connecting to a centralised database running MySQL. This way, both sides of the group could conform to the same coding principles and problems could be shared and understood more easily, overcoming the language barrier.
Figure 1. System for project
To ensure there was an integrated user experience over the whole system, branding and colour schemes were kept similar to convey unity in the project.
User Interface
With the main features highlighted, the system was then built to facilitate demands in the least data intensive way possible with all relevant information provided. With a simplistic approach, the user was to be greeted with as much information in a concise a way as possible.
Figure 2.Basic Initial design for website
Valuable information was to be displayed in each tab in order to separate data so the user did not feel inundated with information. From the start, it was the teams aim to produce a working website (as stated in the Client profile, this was imperative for the Chinese team’s grade) and as such, functionality was always put first.
From this inception, a site map was formed and adhered to throughout the process.
Figure 3. Site Map – Visual representation created by Max Atkinson (Stars represent Group Admin Actions)
With the main design set, the team began building the site in HTML to get a good grasp of the layout and functionality needed. One of the key issues brought up was that the website had to be portable, and as more people use their mobile phones to access the internet it was crucial from a UX perspective to design the web application to work well on mobile. This would require a framework which would make the website responsive to the screen environment. To achieve this we proposed to use Bootstrap which was a free popular HTML, CSS and JS framework used for responsive design. This allowed the website to be viewed efficiently on all portable devices outside the Android app.
Figure 4. Initial website with responsive design
As functionality was developed, the colour scheme was also taken into consideration; chosen in collaboration with the Chinese students. The colours of the app were green to correspond with youth mixed with darker tones to signify calm. The website was chosen to be blue to don a business acumen and display better on older screens.
Figure 5. Final Android app UI and Website
The front page of the website followed a vertical scrolling story, documenting the different functionalities of the website flourished with an animation at the top to grab the user attention. As the android application was built to be a companion app, it was more subdued, ensuring
functionality and branding was uniform. The icon was circled with this idea in mind to convey unity over the whole project, with clusters identifying as groups.
Figure 6. Study with me Icon
Validation
User experience was always prioritised throughout the design process and as such, if any mistakes were made or invalid information was entered, the user was informed straight away. Such instance can be observed during the signup from. For example, username and email were unique fields within this form and as such, two members cannot have the same values. An example of instant validation can be seen in figure 7.
Figure 7. User Validation on Signup form
Another aspect of validation came in the form of animation notification. One of the problems presented when launching in China was the connection speeds. We were told beforehand that the speeds would be slower that the UK but we did not anticipate how slow they would truly be. When researching average speeds, in many places, China was ranked faster than the Uk. OOKLA speed tester provided the speeds based in figure 8.
Figure 8. Screenshot from http://www.netindex.com/download/2,88/China/ - retrieved 17/06/2015
From these findings, we concluded the speeds would not be an issue but on arriving, our speeds were much slower than expected averaging in at about 0.17Mbps for each day of development. This was drastically slower than expected and was due to the extreme population density2. Hence even PHP was slow to react at points. To combat users double clicking options, animations were shown on
2 Land Area – 7,446 Km2, Population – 9.19 million - http://www.chinaknowledge.com/CityInfo/City.aspx?Region=Central&City=Zhengzhou
click to ensure the user knew their command was accepted and processing. This can be seen in figure 9.
Figure 9. Animation Validation example
Platforms – Android Application
One of the main specifications was to have the system in a portable and hence the inception of the Android companion came to light. Although the site was built to provide compatibility with smaller screens, using a native app. This stopped excess data from being downloaded and added extra features for the user.
The Android design was predominantly built alongside the site map seen in figure 3, but as it used the same functions, was able to deviate slightly to entice users to download the app alongside using the website. The main difference on opening the application was the greeting screen. Here the user was met with a tag cloud so they could browse random groups on the go. This was intended to add a different aspect to the system and open the borders for adopters of the ‘Study with me’ project.
Figure 10. Android Tag Cloud
The other big draw to the android app was the timetable creator. This allowed the used to build their own timetable, inserting external data, and display it as a widget. The widget cold be seen and set on the user’s homepage and viewed at any time. An example can be seen in Figure 11
Figure 11. Timetable widget with two classes
To endure the design followed the same pattern as the website, a flow chart was devised to keep development along the right tracks and can be seen in figure 12.
Figure 12. Android Flow Chart
Implementation
Programming Languages / DB analysisMA In Scotland, the Chinese side of the team proposed using Oracle as our DBMS with a Java (JSP) back-end. It was clear that members on both sides of the team were unfamiliar with JSP, not to mention that Java is better suited to medium to large-scale applications and finding a webhost for a Java server is more expensive than most other (more lightweight) languages. Java also imposes a longer development time on the project. Most modern start-up businesses develop with a language that can be deployed fast (such as PHP or Rails) before eventually migrating to the JVM when the application is no longer conveniently scalable.
With this in mind, the group decided to use PHP as the server-side programming language. This choice came because members on both sides of the team were familiar with it – a vital benefit considering the application would be remotely collaborated on. As for the DBMS, PHP provides PDO (PHP Data Objects) as standard in any installation. This offers protection (via prepared statements) from SQL injection attacks when connected to a MySQL database. MySQL is an open-source DBMS becoming ever more popular in modern cloud-based infrastructures, making it an obvious choice for our database.
DBMS/ERD Design Shortly after deciding upon a DBMS, a rough ERD was sketched up to facilitate the basic functionality of the application - Appendix 5.0.
As implementation continued, the database structure was expanded upon as needed. Tables for functions like notifications, endorsements, room booking and tags were added to produce the final ERD, which can be found in Appendix 5.1.
Implementation of DB
Once the group had finalised the database structure, data was needed to populate some of the tables. Data that was added at the start of the project includes the two universities (ZZULI & Napier), along with the classes (modules) and rooms at each university to allow students to book these for meetings.
Napier’s data was acquired in JSON format from the Napier Tracker website with the help of Andrew Cummings. This data was then looped through and SQL insert statements were created and executed in phpMyAdmin. A similar approach for data deployment was intended for the Chinese university; however over time it became apparent that data procurement was not possible for ZZULI. In light of these facts, ‘dummy’ data was pre-emptively created by Tristan in order to supplement the database’s initial data.
Later in the project, it was proposed by the Chinese side to remove all numeric key identifiers from the database and make each primary key a composite key. After discussions in meetings, the group decided to keep the numeric primary keys because removing composite keys is a requirement of normalisation at 3rd normal form. Doing this would also cause complications this with joins and impose a bigger latency overhead – string comparison is slower than integer comparison.
Server Development, testing and EnvironmentSystem architectures
ANAs any Information System project, a server architecture needs to be set up and deployed. The goal is to both provide a place for the web application to be accessible from, as well as providing the backend for the mobile apps. We also need to provide a development platform to enhance the collaborative efforts of the group and securely store any work done.
During the design phase, we had to decide where to host the platform, these were the options:
- Zhengzhou University of Light Industry server. o Dedicated hardware leading to a powerful server. However no virtualisation solution
is installed on it and the network connection is not as consistent as from the UK. The server is sometimes used for student projects which can make it inaccessible because of an un-voluntary misconfiguration: this was experienced during the project, where the SSH access was cut for two days.
- In Napier’s cloud (Dynamic Forensic Evaluation Training3)o It is a virtualization platform (VmWare ESX) based in Napier University. This option
has been put aside as our service needed to be publicly available 24/7. As connections are being managed by the network JANET, strong regulations are setup. To ensure compliance with these regulations would be difficult and time consuming.
- Amazon Web Services cloud computing service. o This solution offer high scalability for the application, quick and fairly easy setup and
very high availability. However this solution has a monthly cost. Where the others solutions have an upfront cost (buying the hardware) that had been made already.
We chose to use as a primary host the AWS solution with some components of the ZZULI server.4
As AWS offer the best reliability (mainly on the access of the service its self), limited policy to comply with and it is one of the fast growing industry technology.
3 http://www.d-fet.eu/4 https://www.ja.net/ - No reference in text
Figure 13. Overview of Amazon Web Services
Development platform
Why Git?
Git is a version control software, which is very widely used for coding and development software. It has been developed originally by Linus Torvald5. We will use our own GIT server to be able to control all the development, to get more features than a shared platform (such as github, bitbucket...), to assure a better privacy and to control the backup.
Git has now become industry standard -with supervision6- when it comes to development and version control software. Most of the team had never worked with it before but a quick training session and some hands-on practise brought everyone up to speed with the basic features.
GitLab
When we speak about git many people think straight away of github but this is a shared service with a limited control over the backup and security aspect of it.
After some research gitlab7 seemed to be a good way to go as it is software wrote in Ruby that sits on a Linux server and acts as a git server with a nice Web User Interface. It also builds all the
5 http://en.wikipedia.org/wiki/Git_%28software%296 https://subversion.apache.org/7 https://gitlab.com
dependencies needed. Behind the scenes, the information is stored in a postgresql database which required some adjustments to get it working such as those detailed below:
Deployment specifics configurationHttpsFor obvious security reason we chose to enable -and force- the use of HTTPS for an end to end encryption of all the data from the user to the server; especially during the commit. The certificate is a self-signed certificate which would not fit a production environment but did the job in a project like this. More details can be found in the security part of this report.
MailThe mail server that comes out of the box with gitlab is postfix; a well-known and really well documented mail process on linux. The initial configuration does the job but not at our expected standards. The emails were classified as spam and not from the address we wanted to use. To avoid being classified as spam and to get our domain to be spoofed, we deployed the following standard industry practices:
- A Sender Policy Framework8 (SPF). A SPF record is a DNS record that states the ip addresses that are allowed to send an email for the domain. For this situation, it gave:
It is a rather simple protection, relatively effective but can still be fooled by forging the SMTP packet with the ip of the domain authorised sender. It was hoped, a second mechanism would exist.
- A DomainKeys Identified Mail9 (DKIM), which uses the same principle as HTTPS using a public/private key.
a. You deploy the dkim key on your Domain Name Server (the public key).
b. In your mail server configuration:
i. Install the dkim module (opendkim and opendkim-tools). And configure accordingly.
ii. Modify your mail server configuration to send the email to sign to the DKIM process listener.
8 http://en.wikipedia.org/wiki/Sender_Policy_Framework9 https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy
c. Check that the emails are being signed by sending a test email.
The strategy we used has prevented us from getting our emails classified as spam but also ensures that our domain can hardly be spoofed.
Result on an email sent to a google client of who has the reputation of being hard with domain:
“Received-SPF: pass (google.com: domain of [email protected] designates 52.16.***.*** as permitted sender) client-ip=52.16.***.***;
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of [email protected] designates 52.16.***.*** as permitted sender) [email protected];
dkim=pass (test mode) [email protected]”
Backup
To ensure our data are safe and secure and recoverable in case of disaster we setup a dedicated backup process that worked as follows:
Integrated WorkflowPrincipe
To beat the time difference and to avoid having to manually propagate the change made to the web project, it was necessary to get an automated system to synchronize it. We had the following requirements:
- Secure as much as possible
- Easy for the user -the developer here-
- Avoid manual action
How is it achieved?
GitLab comes with a very smart way to get customise actions using a hook. It is basically a script that is triggered by a git event (commit, commit received...).
The hook is triggered when the git has properly received a commit to the web project. It then connects with SSH to the web server, deletes the old repository, clones the new version, adds the database connector where it has to be and exits the ssh connection. The idea is rather simple, but tricky to implement.
Application platformWebserver
Apache
The web server is a classic apache2 (2.4.7). It is running on an ubuntu 14.04 LTS.
Virtual Host
The development server is configured to work with virtual host, so we need to have two of them:
- The phpmyadmin virtual host that is setup to manage the mysql database.
- The web.igp.noel.me.uk which host our website.
Virtual host offers a lot of flexibility and enhance security where the directory root directive is set and cannot get up to it.
Https
For privacy, and securing personal data we setup the server to only support https communication. When a communication arrived in http, an http 301 code is answered to redirect to the https version of the website. This is done using the directive “Redirect Permanent” which is considered better practise than using mod_rewrite10.
10 https://wiki.apache.org/httpd/RedirectSSL
To ensure the authenticity of the server and avoid certificate error message (as we get on our git server or our php my admin console), we purchased a Certificate Authority signed certificate.
More details on the security consideration for the webserver can be found in the “Security features” part of this report.
Caching
To get a faster loading time on the website we enabled a caching directive.
Apache has a set of directives to indicate what should and should not be cached by the browser client and how long they should be cached for.
We use this for when the module “expires”. We then set the type of file that should be cached and for how long as shown in the example below:
ExpiresByType image/png "access plus 1 weeks"
It give in the header:
DatabaseMysql
The database we used is a Mysql database which brings as a Software As A Service (SAAS) by Amazon Web Services using the service Relation Database Service11 (RDS). This service enabled the ability to deploy a quick and very reliable database with the classic feature of a DB already deployed (Backup, access management, possible high availability and redundancy). For security reasons we only allow on the firewall sitting in front of it the access on port 3306 from the web-server and the phpmyadmin server.
11 http://aws.amazon.com/rds/
Phpmyadmin
To simplify the management of the database we installed a phpmyadmin server. It in fact, is on the same server as the webserver but is configured with a different a domain name (https://phpmyadmin.igp.noel.me.uk). The configuration has nothing particular that is worth describing here.
Security features
Any information system has to be conceived with security as a primary concern in mind; our project is no exception. The underlying infrastructure is the first part; ensuring the application works on a reliable and secure environment. This part describes the specifics elements we deploy for it.
Https
Https was systematically enabled on our different site:
- phpmyadmin.igp.noel.me.uk
- git.igp.noel.me.uk
- web.igp.noel.me.uk
However only web.igp.noel.me.uk was used with an authority signed certificate for cost reasons, it is also the only one that should be publicly accessible. The other sites were setup with self-signed certificates and were only accessed by the development team.
Https provide us with:
- Confidentiality; as the communication is encrypted between the client and the server.
- Integrity; which is handled in both http and https by the Transmission Control Protocol12
- Authenticity; it mainly reassures that the web.igp.noel.me.uk uses a publicly recognised certificate. It is also true for the others if the self-signed authority is added as a trusted CA.
Ssl configuration
Through the recent months a number of important attacks have targeted the openssl software and a number of cipher. To prevent being susceptible to this attack, a number of counter measures had to be implemented on the SSL layer on the server.
1. Disabling SSL V2 and V3 that are now considered as weak13
2. Only accepting a certain amount of cipher in a precise order from the prefer to the non-accepted one:
3.
12 https://msdn.microsoft.com/en-us/magazine/cc163827.aspx13 https://blog.mozilla.org/security/2014/10/14/the-poodle-attack-and-the-end-of-ssl-3-0/
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5
1. Checking the configuration using industry standard tool www.ssllabs.com :
fail2ban
Fail2ban14 is a piece of code that is made to prevent brute-force attacks and Denial Of Service against services on a server. We used it to prevent the SSH access from being brute forced by setting it up so that after 3 tries of a password the ip will be blocked on the firewall for a certain amount of time. This technique helps to temper a lot of brute-force attacks. We needed this tool because we were not able in the first place to restrict the SSH access to a couple of IPs. This is normally done by using a VPN access but we were not able to set one up from the beginning because of time constraints.
Pemkey
To access the different server with SSH we choose to use access key of 2048 bytes. This is a very good protection against attacks as the key is way longer than any password realistically usable. The drawback is that the key has to be stored securely and in the case of loss; the access can be definitely lost. This technique is also really useful for accessing server in a script as no passwords are needed; just the permission key has to be provided.
2 factor authentication
PrincipeTo harden the security of an authentication factor (password, pem key, token...), we use one of a different type. For example, some banks will ask you for a password (something that you know) to connect to your bank account and then they will send you a text message or use a token on a different device (something you have).
14 http://www.fail2ban.org/
Nb: It exists three type of authentication factor(https://technet.microsoft.com/en-us/library/jj713614.aspx):
- Something you know (password, public key)- Something you have (a phone, a laptop a physical MFA)- Something you are (biometrics, fingerprint)
The biometric factor is still difficult to implement on a server side, but the something you have is rising more and more. More and more service proposes, such as for a Google Account, a facebook account or an Amazon Web Services account.
How is this useful for us
Our git server being the key point of our architecture, it need to get a secure as it could get. However, it should not be enforced on all the incoming ssh connections as the developers can use SSH to push/pull the data with the git server. We only enabled the multi-factor authentication for the administrator of the server (member of the group “sudo”)15. For convenience we also disable the two steps authentication from a known IP (personal VPN server).
How did we implement it
We used the Google Authenticator project16 that is implemented into the “Pluggable Authentication Module” that is then called by SSH. To get the time based token, the authentication token is retrieve using the application Authy17.
1) Install the Google-Authenticator module.
2) Modify the /etc/pam.d/sshd config by adding the following line:a) Add the google verificator, MFA set to sufficient so doesn't ask a password but
(a) if google-authenficator is not set it will ask password(b) auth sufficient pam_google_authenticator.so
3) Modify the /etc/ssh/sshd_config:a) Change to yes to enable challenge-response passwords (beware issues with
some PAM modules and threads)ChallengeResponseAuthentication yes
b) Enable two steps auth for group sudoMatch Group sudo Address *,!167.abc.abc.133/32AuthenticationMethods publickey,keyboard-interactive:pam
15 Based on: https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-two-factor-authentication, http://www.linux-pam.org/Linux-PAM-html/Linux-PAM_SAG.html and http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/sshd_config.5?query=sshd_config&sec=516 https://github.com/google/google-authenticator/17 https://www.authy.com/users, Authy offer the advantage to be mutli-device. It improve the reliability has several device can get the verification code at any time. It is also one of the market leader in MFA.
4) For the user you want to enable it for, it will ask you a couple of question to answer depending your need:a) > Google-authenticator
b) You will receive your authorization code (blurred for security reason):
5) Restart the ssh service
6) You will be then prompted for your verification code
Patching
Regular patching of the instances was planned to prevent against any security holes, only the git-lab component (so the git-lab as a service) has not been upgraded for stability reasons but also, no security release has been made on our version.
From development infrastructure to production/demonstrationRequirements
Context
The system that has been built so far presented a lot of advantages that fully answer the previous requirement. However, for the presentation that takes place in China, further issues have come to us that force us to design a specific environment for it.
The problems we are facing
- Lack of flexibility in the development environment (the firewall are being closed as much as possible from external ip such as for the SMTP server or the Database connection).
- Very limited scalability as the environment is not directly controlled by use and the instance heavily rely on each other.
- The development webserver is made to handle the load of the development team (13).
- The servers are in Ireland which give a high latency from China, which could make the client to decide to drop the use of the service.
Needs
We want to:
- Reduce the loading time
- Even the user experience wherever his location
- Improve the redundancy of the server.
- Get a dedicated production environment.
Overview of the changes
Only the web delivery needs modification to handle the load. The delivery of the web application required three components:
- A MySql database to store the data
- A web server to serve the content
- A mail service to send the notification emails from the application.
We will also study the implementation and the needs for a caching layer in front of the website to eventually speed up the request to the web server. This part, if implemented is an addition and not a change.
For more clarity, the production environment will be map under a different domain name (*.igp.noel.me.uk for the development environment and *.atux.co.uk for the production environment).
Architecture
Note:1) Regions/availability zones18:
i) A region in AWS is a geographic zone where the services are. For example ap-southeast-1 is in Singapore whereas eu-west1 is in Ireland.
ii) An availability zone is situated in a region and is built for redundancy so for example ap-southeast-1a, ap-southeast-1b will have different electric arrival, different internet access. They are built to maximise the distribution of a resources
iii) Depending of a service they are either in an availability zone (Elastic Cloud Compute), in a region (Elastic Load Balancer) or global in the different Amazon edge location for example CloudFront in HongKong.
2) Represented flows the sketch only represent the flow that we manage directly and that we setup.
3) Infrastructure as a service (IAAS) and Software as a service (SAAS):i) IAAS:
(1) Elastic compute Cloud (EC2), resources allocated to a server.ii) SAAS, no access and no need to maintain the underlying infrastructure:
(1) Relation Data Base (RDS) a database is installed and maintain for you, you only manage the mysql application.
(2) CloudFront, proxy/cache layer distributed over the world. You manage the caching settings
18 http://docs.aws.amazon.com/general/latest/gr/rande.html
(3) Simple Email Service (SES), an SMTP host that manage the ip reputation and routing for you.
(4) Elastic Load Balancer (ELB)(5) Auto Scaling Group (ASG)(6) Route53 (DNS service)
Domain Name service
For the webstack we want to reduce the latency as low as we can. Route 53 is the DNS hosted by AWS. It supplies a DNS distribution over the world. It also offers a very interesting option in the IP to be served depending of the name you request. You can for example, serve different IP depending on the location of the requester for the same name. We have not used this possibility here, but it would be really interesting if this project was to go live to serve an European environment and a Chinese one for improving loading time.
Mail sending
The postfix we set up previously worked fine, however there is a lack of scalability as it is tight in the git server, also postfix need to get each ip to be explicitly allowed to send email. In the hypothesis of scale up/down of our number of instance this will create trouble.
We choose to use Amazon Web Service Simple Email Service which is a highly available very scalable SMTP relay host. It is rather simple to setup in a few steps:
1. Verified the domain you want to use (@atux.co.uk)2. Verified the address email you want to appear in the sent from field. This steps is made to
protect against email spoofing. 3. Add the DKIM record to sign the outgoing email. 4. Test that the behaviours is as expected. 5. Asked the Amazon to leave the sandbox19.
Database
ImplementationThe database we used before is in the Ireland region. To speed-up the requests, we moved the database to Singapore region, we enabled the multi-AZ options to improve reliability. The methods give us more control of the settings, especially the software firewall sitting in front of it. With a tag on the web instance (the security group here), they are automatically allowed in the firewall. This system make reliability and scalability automatic and secure as only the current IP used by the web servers are allowed.
We created a dump of the relevant database and upload it to the new server. We then enabled replication master/slave on the mysql application.
19 When you initiate you a new SES, you first setup in a sandbox. It means you can only send email to verified recipient. This is setup to test your application and to avoid it to leak (start send spam email cause of a misconfiguration). When you fully setup, AWS will allow or not your demand to use their services.
In the meantime to avoid confusion, the phpmyadmin, and the database connector on the web server point to the new database.
Testing
We had issues on the connection with the phpmyadmin caused by permission and upgrade as we migrated the database data and table but have not replicate the mysql table (mysql.user in particular) as it can cause conflicting configuration.
Production webserver
The production servers get the same settings on the application layer that presented before. The infrastructure layer is however different. Only the difference will be detailed here.
Ami (Amazon Machine Image)
The workflow described before the development environment with an automatic deployment in production and is not a good practice for a production system as any code change could impact it. An instance automatically receives the new code version. When the code or system modifications are considered ready for deployment, an image is taken of the server (called AMI). The image is then set in the production environment as explained below.
This workflow restricts the ability to change anything in the production environment without manual action; also it is interesting for patching/rebuilding a server. You can patch the image and then deploy the patched image in production. It allows for one patching no matter how much image of the server run.
Automatic scaling groupThe ASG serves two purposes:
- Managing the production version as explained below.- Coping with increase/decrease of traffic.
Managing the production version (Launch configuration)
You determine the image you wish to use for your instance, the instance type (the size)… You can also determine specific tags on it and the firewall that sit in front of it (Security Group).
When a new image is ready to go in production, it is just needed to create a new launch configuration.
Scaling policy
A scaling policy is composed of:
- The launch configuration you want to use (so the image)
- The number of instance you want to be running (to do a production release, you change the launch configuration, double the number of instance and then bring it back to the normal number).
- Your scaling decision: You determine in which case (metrics such as CPU, memory, or application trigger) you should increase or decrease the number of instance running.
o The range of number of instance you wish to have running (minimum, maximum, desired).
o The “health-check”, a test to confirm the instance is running fine.
These settings give us two instances always running with a very low probability of getting a complete loss of capacity.
Elastic Load BalancerPrincipe
As we get at least two instances running with the settings allowing up to four; we need a way to split the traffic as evenly as possible to get the server running equally. More than just routing the traffic, we want the load balancer to be dynamic. It has to deregister not healthy instance and to register the new healthy one.
A common way to distribute the traffic is to load balance using DNS and round-robin. This system is effective however; it doesn’t integrate very well with quick up/down scaling as DNS record need time to propagate.
Implementation
We used Elastic Load Balancer, which a service proposed by Amazon. It is made to interact with the other AWS components and therefore make the implementation rather straight forward.
1. Create load balancer2. Define forwarding policy3. Register the ASG in the load balancer.
Issue:
While we setup the load balancer we encounter two issue linked to the web application:
- The traffic is being routed equally to each instance no matter of previous interaction. However the phpsession stored in the client browser are stored locally on the server as per the php code. If the session is not routed to the same instance it can cause the session to be lost and to have to login again.
o We enabled a special cookie on the load balancer that ask it to route a session to the same instance to avoid a new login process.
o Resolved- If one of the instances is brought out of the load balancer, no traffic is being routed to it
anymore but your browser still contains a session for it. The load balancer will send your traffic to another instance to maximize time. However, the other instance doesn’t know the php session you are giving in your cookie; that causes a loop in the application.
o To solve the loop a simple page refresh stop it and bring you back to the homepage. You however have to login again.
o The solution will be to store the session id in the database and to get the server requesting so it avoids the need to login again. This solution would have solved the previous issue as well.
o Not implemented because of time and probability of this situation being rather low for our presentation environment.
Proxy/caching layerPrincipeWe were wishing to implement our self a proxy server in China using NGINX. However we have not found any cloud provider capable to provide instance in China for an equivalent price of AWS. (We had a look at aliyun which is a concurrent of Amazon in China however, the only propose their service in Chinese, at the date of redaction of this report. This would have made the configuration difficult.).
We decided to use cloudfront for our proxy/cache layer. Our choice to use this service was driven by their nearest location in HongKong, but also by the distributed location all over the world which enhanced the browsing experience all over the world.
Overview of the flows
Testing
To estimate the efficiency of our system and the improvement made. We download the webpage from 5 different servers around the world.
Methodology
- Downloading 5 time the welcome page with wget.
- Get the average bandwidth out of it.
- Using 5 servers around the world (Ireland, France, Singapore, China, and Canada).
- Taking Edinburgh Napier, Zzuli website as comparison.
Important note:
The number has to be handled carefully as our webpage is rather light and the average bandwidth measured may not be representative of the user experience. However, it shows clearly the difference between our system and the improvements that we made.
To get data more representative of the user experience we did a comparison using chrome and its developer tool (network tool, data “Finish:”).
Results
Interpretation Our number shows that the improvement we made really helped to improve the loading time of our website. On average a simple web instance (in Ireland or in Singapore) has the same throughput over our testing. However our production environment managed to pull up to 2 and half time the speed of a normal instance. This number seemed to be confirmed by the loading time measured with Chrome even if the sample is smaller, with a loading time drop by at least half.
Data analytics:CloudFront come with a very interesting data analytic dashboard. It provided information’s such has20:
- Type of traffic (http/https)
- Most requested object and http code answer of it
- Location of the requester
- Browser used…
Way forward
We could imagine a scenario with the application having a great success in UK and in China. In this case we could deploy the same environment in the Ireland region (automated by creating a template) and balance it using the geolocation of the source ip in the DNS request. Furthermore we could improve the cost efficiency of the infrastructure by reducing the auto-scaling group to only one instance by night and to spin-up a second one early morning with a script using aws Command Line Tool.
Pricing
It is difficult to estimate the price it would cost to run the current production architecture, as part of the price is the data transit. We could however estimate a cost between $150-250 a month by running only on demand system and rather powerful instance in the auto-scaling group.
After a couple of months of running the system you could get an accurate idea of the resource you need to run to cope with the traffic. When you know the instance type you need, you can reserve the resource for 1 year or 3 year and pay the bill up front. This can bring down the cost by up to 75%21.
Language Framework Introduction MA With PHP selected as our server-side programming language, research began into which framework to use. The PHP SLIM framework provides great functionality for implementing a RESTful API, allowing the programmer to define routes and return appropriate data from the database. For these reasons, SLIM was selected for this project.
As for the client-side, the bootstrap framework was used to create a visually stimulating UI, along with jQuery for basic DOM manipulation. Initially, the Backbone JavaScript framework was considered although this turned out to be unfeasible as many members of the group were unfamiliar with web development in general so diving into an advanced JavaScript framework would require a lot of learning – an inconvenience for the project deadline.
20 For a full list of the feature available: http://aws.amazon.com/cloudfront/reporting/21 Sources: http://aws.amazon.com/ec2/purchasing-options/reserved-instances/
PHPNH As the application was data-driven, a connection to a database was required. To achieve this we chose PHP. PHP is a server side scripting language widely used and suited language for web development (PHP Group, n.d.). This allowed easy and secure connections to a database and produce dynamic web pages. Alternative languages were considered such as Active Server Pages (ASP) and Java Server Pages (JSP). A list of advantages and disadvantages are listed below, these helped with the decision of choosing PHP.
Advantages
- Open Source – This allowed us to use PHP without purchasing a license.
- Huge User base – Installed on over 240 million websites (Netcraft, 2013).
- Support - Large communities of support which made learning the language and solving problems easier.
- Cross Platform – PHP supports various web servers such as Apache and Microsoft ISS.- Built in database support – Libraries such as PDO allow easy and secure connections to any
type of database (PHP Group, n.d.).
- Simplicity – PHP is procedure programming language but also includes object oriented programming features. Making it again easier to learn and implement.
- External Libraries – external libraries such as email, PDF, graphs etc. are widely available and usually free for use.
- Speed – PHP is fast to execute
Disadvantages
- Security – As PHP is open source anyone can view the source code. If a bug was to be discovered this would cause a security threat.
- Not suitable for large applications – This depends on how the files are managed, however the project isn’t a large application.
PHP Development Approach
For the project different environments were used to build and test PHP. These were local and live development, both having their own advantages and disadvantages.
- Local DevelopmentSoftware such as EasyPHP or WAMP was used to build PHP functions locally before being
pushed to the Git server. This had the advantage of speedy development and testing especially with low internet bandwidth which was experienced in China. Any critical errors were contained to the local computer and did not affect the live server. A disadvantage of this was not having the feature of version control which meant any accidental loss of files could not be recovered. Also care was required when pushing new code to the live server as sometimes more than one developer worked on the same file.
- Live DevelopmentThe Git Lab server this allowed all the developers to have an account to push new files and make changes. Some development work was done straight to the server. Advantages of this were fast implementation and version control features. Disadvantages were connections speeds could affect work load, errors occurring from at least two people working on the same file and knock on effect errors from common functions.
PHP Prepend
The prepend PHP file is included in all web pages and PHP function before any executed code. It is the configuration file and contains the database connection, starts PHP session storage and instantiates the student class.
PHP session storage is used to store the unique database ID of a student that is logged in. That ID is then passed as an argument when instantiate the student class.
Student Class
The student class always instantiated in the prepend file, it either takes an integer value (unique student ID) as an argument. The constructor of this class uses this integer to find the student in the database and store all the information in the class. This class has methods such as userExists to determine if a user is logged in.
The student class is always instantiated in the prepend file this accepted an integer argument (unique student ID). The constructor of this class used the integer to find the student in the database and store all the information in the class. This object was accessible in all PHP files which
allowed for faster development and no duplication of code. Example of this was using the method userExists to determine if a user was logged in. (see figure 2)
<?phpif(isset($_SESSION)){
session_start();}// Declare Student Class$sessionID = 0;if(isset($_SESSION['studentID'])) {
$sessionID = $_SESSION['studentID'];}$student = new Student($sessionID);// Check login status$student->userExists(); /* returns true or false */
Figure 14 – User Exists Method
PHP Slim API
PHP Slim is a PHP micro framework that allows developers to make simple, powerful web applications and API’s (PHP Slim, n.d.). The PHP Slim framework was used to create a RESTful API service to be used by the web application and the Android mobile app. The API was stored in the scripts folder and worked by GET and POST HTTP requests as seen in Figure 5.
Figure 15 - Get/Post HTTP
JSON (JavaScript Object Notation) was used to transfer data to and from between the server and the client. JSON is human readable and is readily available in many programming languages which made it an ideal data interchange. This made it possible to use one framework for two different program languages.
PHP Slim works as a router taking into consideration the URL and the HTTP request type to fetch a method or function. CRUD functions were stored in separate files and were included into the index file to keep the file structure organisanised and easy to manage.
An example of this is a POST request to “~/scripts/student/register” which is used to sign up a student. As seen in Figure 6 the route is declared and includes the addUser.php file and calls the function. Figure 7 shows some of the code in the addUser function, the POST data is the retrieved by Slim.
<?php// Require SLIM filerequire '../Slim/Slim.php';// Require prepend file – contains SESSION, DB Connect etcrequire 'prepend.php';// Instantiate Slim Class\Slim\Slim::registerAutoloader();$app = new Slim\Slim(); // start it up and declare our routes// Route for adding student$app->post('/student/register/', function () use ($app){
include 'addUser.php';resetPassword();
});// Run Slim$app->run();
?>Figure 17 - PHP Slim Sample Code
<?php
function addUser() {global $db;global $noerrors;
// If database connect details are incorrectif($noerrors <> 0){echo json_encode(array("result"=>"Failed","error"=>"No
Figure 16- Route
connection")); return;}$request = Slim\Slim::getInstance()->request();$body = $request->getBody();// Fetch Post JSON and decode$jsonData = json_decode($request->getBody());
Figure 18 - addUser.php
PHP Slim DocumentationAs the other group were responsible for the development of the Android app, documentation was created for all the API. This was to ensure a smooth connection from Android and PHP Slim. This included the format of JSON objects for sending to the sever, expeced return values and error codes. An example of this documentation can be seen in Appendix 6.0.
PHP Coding PracticeWhile coding for PHP the developers ensured good coding standards were met. As the group had four developers working on PHP it was crucial that good practices were made. This ensured that tasks were completed in a timely manner and development errors were kept to a minimum. These practices included:
- Comments - All functions and parts of code were fully explained to what it did.
- Naming conventions - Camel Case was used to describe all variables.
- Simple code – Complicated code was avoided so each developer knew what code was used for. This avoided future errors by misunderstood code.
- Portability – Configuration files such as the prepend file never contained any hard coded values for example connection credentials for the database were stored in global variables. Relative links to files were used instead of absolute paths, this was necessary for local and live development.
- Minimised duplicated code – functions were created where code would likely be used more than once, to ensure fast development and performance.
These standards were also adhered to and maintained when writing in JavaScript and across the project in general.
PHP Security
$firstName = '';$email = '';$university = 0;if(isset($jsonData->first_name)){
// Remove any html characters$firstName = htmlspecialchars($jsonData->first_name, ENT_QUOTES);
}
if(isset($jsonData->university)) {// Ensure University ID is an Intintval($university = $jsonData->university;)
}if(isset($jsonData->email)) {
$email = $jsonData->email;}
// Ensure email is validif(!filter_var($email, FILTER_VALIDATE_EMAIL)) {
$errors[] = "Valid Email Required";}
As this project was web based and used an API service (PHP Slim), web files were completely accessible over the internet. This required us to code securely to prevent SQL hijacking, system errors and any other problems. The approach taken was assume that all data posted to the server was not legitimate. Validation was used on each variable to ensure only the expected value was accepted. Figure ({0}) shows sample code of the AddUser function - ‘htmlspecialchars’ function was used on certain strings to prevent JavaScript injections.
Along with validation on all variables, measures were taken to prevent any SQL injections. PDO (PHP Data Objects) was used to prepare all statements made to the database. Each query made was prepared to ensure no SQL injections could be made as seen in figure ({0}). These statements have not been tested for 2nd SQL injections but ensuring prepare statements were used throughout the application minimises this risk.
// Check if email already exists$emailQuery = $db->prepare("SELECT * FROM `students` WHERE `email` = :email");$emailQuery ->bindParam(':email', $email);$emailQuery ->execute();
// INSERT STUDENT SQL STATEMENT$insertQuery = "INSERT INTO `students`(`dateJoined`,`universityID`, `schoolID`, `first_name`, `last_name`, `DOB`, `email`, `username`, `salt`,`secure_password`,`type`) VALUES (CURDATE(), :universityID, :schoolID, :first_name, :last_name, :DOB, :email, :username, :salt, :secure_password, :language)";
$q = $db->prepare($insertQuery);$q->bindParam(':universityID', $university);$q->bindParam(':schoolID', $school);$q->bindParam(':first_name', $firstName);$q->bindParam(':last_name', $lastName);$q->bindParam(':DOB', $dateOfBirth);$q->bindParam(':email', $email);$q->bindParam(':username', $username);$q->bindParam(':salt', $salt);$q->bindParam(':secure_password', $securePassword);$q->bindParam(':language', $language);$q->execute();
To prevent system errors and other problems error catching was used and custom error messages were implemented. Figure A shows an example when core variables failed the initial validation, a custom error message is outputted in JSON and running code halted.
// Check date and time are in correct format$dateBits = explode("/",$date);$timeBits = explode(":",$time);$validateFormat = false;if(!(COUNT($dateBits) == 3 && COUNT($timeBits) == 2)){echo json_encode(array("result"=>"Failed","message"=>"invalid date format"));return; /* Ends function */}
// Check date is validif(!checkdate( intval($dateBits[1]), intval($dateBits[0]) , intval($dateBits[2]) )){echo json_encode(array("result"=>"Failed","message"=>"invalid date"));return;} /* Ends function */
As the web application involved users creating their own accounts it was essential that we used effective encrypting methods. To achieve this we avoided using MD5 hashing. As in the scenario of a security breach, existing rainbow tables would be easily used to decode the user’s passwords. Instead we used SHA-2 (Secure Hash Algorithm) and a long unique Salt for each user to ensure maximum protection. Figure A shows the code the application used to encrypt the password and how it was used for authenticating users.
// Generate Salt and Hash Password$password = $jsonData->password;$salt = bin2hex(openssl_random_pseudo_bytes(24, $cstrong));$securePassword = hash('sha256', $salt.$password, false);
// Login Query$query = "SELECT * FROM `students` WHERE (`username` = :username) OR (`email` = :username)";$stmt = $db->prepare($query);$stmt->bindParam(':username', $username);$stmt->execute();
$row = $stmt->fetch(PDO::FETCH_ASSOC);$salt = $row['salt'];
$secure_password = hash('sha256', $salt.$password, false);
if ($row['secure_password'] == $secure_password){
$_SESSION['studentID'] = $row['studentID'];$returnMessage = '{"result":"successful" }';
}
File Structure
All files on the webserver were organised into separate folders, this was to ensure better management. The file structure for our project can be seen in figure ({0}). The routing feature of PHP Slim was used in the root directory so only the index file was required. This worked by fetching files from the html directory depending on the URL. This kept the root directory tidy and allowed us to take advantage of search engine friendly URL’s.
Rootcss/ < -- Contains all CSS files -- >js/ < -- Contains all JS files -- >fonts/ < -- Contains all font files -- >html/ < -- Contains all HTML files -- >img/ < -- Contains all images -- >ckeditor/ < -- Contains JS files for ckeditor (wysiwyg) -- >email/ < -- Contains PHP mailer scripts -- >Slim/ < -- Contains PHP Slim source files -- >scripts/ < -- Contains all PHP files (configuration, API) -- >index.php.htaccess
ReferencesNetcraft. (2013, January 31). PHP just grows & grows. Retrieved June 21, 2015, from netcraft.com:
http://news.netcraft.com/archives/2013/01/31/php-just-grows-grows.html
PHP Group. (n.d.). PDO Introduction. Retrieved June 21, 2015, from php.net: http://php.net/manual/en/intro.pdo.php
PHP Group. (n.d.). What is PHP? Retrieved June 21, 2015, from php.net: http://php.net/manual/en/intro-whatis.php
PHP Slim. (n.d.). Retrieved June 21, 2015, from slimframework.com: http://www.slimframework.com/
Notable FeaturesNotificationsMA The Study With Me application informs users of activity within their groups using notifications (much like social media sites Facebook and Twitter). When implementing the system, the two database tables used were ‘notifications’ (to store the details about the notification itself) and ‘user_notifications’ (defines whether a user has read the notification or not).
The addNotification PHP function inserts a notification for a group. This function takes in 4 parameters: the notifying student, the group ID, the meeting ID (if relevant) and the notification
type. The function simply inserts a notification into the database and is available to the users. This function was defined within the SLIM router and is available throughout the program, allowing easy dynamic insertion of notifications wherever they are needed.
The getNotifications() PHP function simply returns relevant notifications to the signed-in user (for groups that the user is in). This script is polled in JavaScript using the pollNotifications() function which implements jQuery’s $.ajax() method and is executed every 5000ms from the main dashboard page. On each poll, the number of notifications on the screen is updated if there are any new unread notifications, along with the <title> HTML element (so users can see their notifications when browsing other tabs) – simplistic yet effective UI.
If the pollNotifications() function detects any new unread notifications, they are appended to the table using the populateTable() function. This loops through the notifications and selects the appropriate icon/template depending on the notificationType attribute, before using jQuery’s append() method to insert <tr> elements into the notification table. If the notification is read, the function applies a bootstrap class to dim the colour. The bindNotifications() function is then called to re-bind the jQuery ‘click’ handler to the newly created elements so they can be interacted with when clicked.
To read a notification, the user can click any <a> element within the <tr> of that notification. One of these <a> elements will link to the notifying student and the other will link to the group being notified. When one of these links is clicked, the readNotification() function is called. This sends the notification ID along with the current user’s ID to the PHP function which then updates the notificationRead field within the ‘user_notifications’ table for that user. This also reads ‘similar’ notifications for that user – i.e. notifications of the same type from the same group. The user is then redirected to the clicked page to ‘view’ the notification.
Rating System
CB Throughout the whole process, the most heated aspect of the project was always the Rating system. The Scottish side argued that it was not for students to adjudicate the performance of each other, yet the job of the lecturers. The Chinese side laboured that each person must be rated and group in order to ascertain which group worked best. Systems based on user voting could be skewed and those which boost group ratings based on size were weighted incorrectly and would also be detrimental to the integrity of the rating.
Hence, a completely autonomous system was invented. This was based on three criterions;
• Meeting Attendance
• Endorsements (love from other users)
• Chat participation
The rating system can be expressed as the equation seen in figure 18. It is also to be noted, in order to stop groups from attaining high ratings and coasting once set rating was achieved, all components
(except Endorsements) were reset after 3 weeks meaning ratings worked on rolling basis and constant group activity was needed to stay at the top.
Figure 19 Group rating system
The output from this system is a basic number between 1-100, indicating a percentage. This was then set into 5th, and expressed as a star rating.
Figure 20. 1 Star Group rating example
Android implementation
The android application was mostly completed by the Chinese team. Based on the PHP functions written for the website, the Android app was fleshed out using built-in Android animations and a sliding menu. Unfortunately, a lot of the Chinese work has been closely guarded and ascertaining source code has been impossible with instant dismissal at enquiry with little reason as to why that is the case. As a result, we cannot go into great detail regarding full implementation of the Android app without venturing into conjecture. There are however notable features that we have had direct input into.
MeetingRating=∑❑
Meetings
(meetingAttendancenumberInGroup )numberOfMeetings
LoveRating= amountOfEndorsementsInGroupnumberInGroup∗(numberInGroup−1 )
MessageRating=numberOfUsersPostedtotalNumberOfUsers
Rating=(3∗MeetingRating)+ (2∗LoveRating )+MessageRating
6 ∗100
Notable Feature - Widget
As shown in the design section, the widget feature is based on the time table creator. This allows users to input and conveniently view their university timetables, ensuring they don’t click attend on meetings taking place when they are in a class. Users are presented with a grid in the app. When a square is tapped on the grid, classes can be added. The parameters for adding classes are class name, teacher, duration and room. The widget is then updated – the Android grid (XML) is exported and displayed as a PNG in the widget (See figure 11 for example).
Background Management SystemMA The background management system was something proposed by the Chinese at quite a late stage; the purpose being to monitor users, groups and meetings whilst providing some ‘big-data’ analytics using Baidu’s eCharts JavaScript library (Chinese software) – luckily almost identical to Google Charts. The analysis conducted informs the administrator of the most popular rooms at each university and the most popular major studied by users of the site. Additional functionality offered by the background management system includes the ability to monitor which groups are active and which aren’t. This has various maintenance advantages – groups that haven’t been used in months can be simply shut down by the background manager.
Initially, the background manager was created with experimental data in order to test the eCharts library was configured correctly. Once this was established, AJAX requests were implemented using jQuery to fetch real data in order to populate the graphs.
TestingAs the project was implemented using the agile methodology, there were a few testing methods involved - primarily, Test-Driven Development (TDD). TDD is very useful as tests are written first; forcing the programmer to think more critically about the ‘corner’ cases and small bugs that may arise during implementation – in the end, more robust code is produced.
JavaScript unit tests were created to test the outputs of each function were correct. Test cases were created to test a wide range of data. An example unit test can be found in Appendix 7.0... Similarly, a PHP function tester was created in order to unit test each PHP function. This comprised of a HTML page with a ‘go’ button which would cause an AJAX request to be sent to the script being tested. When the server responds, the script’s output is tested against an expected output and the result of the test is appended to the page.
During the project, unit tests proved useful for fine-tuning and validating input/output within every function; both client and server. However, certain features of the system, such as addNotification (which is called several times in different scenarios) required some integration testing. Doing so ensured harmonious data flow and execution between functions written by different developers.
Once functions were integrated (via GitLab), they were automatically synchronised with the web server and integration tests could begin. Common errors found included type mismatches and incorrect variable names, particularly when data is returned from the server and interpreted by the web browser – this had to do with the way PHP data is encoded into JSON – the PHP variable names are preserved, thus must be consistent with the receiving application.
With an integrated and tested system online, the final stage – user testing – could commence. A Google Forms poll was created in order to find out what users thought of the website, covering features from general usability/site flow to actual functionality. The link to this poll was sent to relatives and acquaintances of the group to receive their feedback and made adjustments based on the feedback received in order to deliver a well-functioning system which also addresses common usability problems.
PHP Function Tester
PK Another aid we employed when developing and testing our PHP functions, was that of a function tester. It exists as its own page within the system, not interacting with the system however. This was a simple function that called the function to be tested, gave it variables and responded to the output of the function. This is similar to unit testing as we are separately from the main code running an individual part of the system. Functions output would be written to the page so you could see directly what was happening. This was essential when trying to debug PHP, as it is run on the server it is difficult to see where mistakes have been made as there is no method for stepping through the lines of code like you would within the console manager for JavaScript. The function could be changed quickly by including the function you wish to test and commenting out previous functions.
Once it was established that the function worked, it could then be added to the system in the relevant way, without the risk that it would break the system.
<html><?php
require 'scripts/prepend.php';include 'scripts/addNotification.php';include 'scripts/readNotification.php';
if (isset($_GET['hello'])) { //if you want to test a function include it above then enter a new line below and comment out the
restaddNotification("New Message",26,92,null);
//readNotification(120);}
?>Hello there!<a href='?hello=true'>Run PHP Function</a></html>
Risk Evaluation/Major IssuesPK Throughout the 3 stages previously laid out, there were many times where we had conflicts and issues. This is very common when working as part of a group. As members of the IGP we had to deal with many extra trials and tribulations involved with working across a language and cultural barrier. All of the problems we faced took teamwork and negotiating skills on everyone’s behalf in order to overcome these, move forward, complete and make a success of this project.
The biggest and most evident issue during the entire project, was obviously the language barrier. The Chinese students spoke really good English and obviously understood it, however the Scottish students had no Chinese and often spoke too fast and in colloquial dialect. This continually cropped up in discussions but was highlighted especially in conjunction with any other problem. Both the group members and our team leaders understood this would be the biggest challenge, to counter this the first week of stage one was targeted at opening up communication between the two groups. At first this was just getting to know each other, but within a couple days this moved onto us trying to explain ideas for the project. This is where we first ran into serious problems with the language. Struggling to explain their own ideas and grasp the idea’s that the Scottish contingent put forward put an obvious strain on the Chinese side of the project. This was never as evident as when we had finished our tasks for the day, we would be socializing in the evening and both parties would get caught up trying to explain ideas in more depth and over again to try and help others understand. As you can imagine this was extremely frustrating for all of the group members. Eventually one of the way’s we solved this initial struggle to understand was by having the Chinese students who understood the Scottish ideas to explain them to the other members of their contingent, and vice versa for the Scottish side.
Another major issue in stage one was the relationship and behaviour of the different groups in the presence of their teachers/lecturers. The Scottish students had a much more professional relationship with their lecturers, whilst the Chinese almost revered their teachers. Retrospectively speaking this is ingrained within Chinese culture, sometimes it is not so obvious but working within this environment it was. They would struggle to speak whilst they were in the room, the level of respect was so high that it would completely change the group dynamic when teachers were present. This meant it was a struggle to gauge a true representation of any of the Chinese student’s feelings. In order to combat this we found it far easier to have discussions and votes when the teachers/lecturers were out of the room, there were several occasions where they were asked to leave the room by the group. This enabled us to have fair votes and open discussions when deciding which topic to pursue.
Related to this was the fact that the Scottish students felt that the Chinese students were being forced to push a specific idea that had initially been put forward to the Chinese side of the group by their teachers. As a reaction to these suspicions the Scottish students had no hesitation to take them to their lecturer. A combination of the Chinese culture, language barrier and not knowing whether this was true or not left the concept open to imagination. It is safe to say that the Scottish students and lecturers imaginations then ran wild coming up with the most ridiculous notions. After a few days to let the dust settle and some deeper communication between the groups including a secret meeting between just the group members, meant we eventually got to the bottom of the issue and could move on with the project. This is an example of how exposure to a little of the Chinese culture resulted in the Scottish students jumping to conclusions and over exaggerating things, which may
actually be a little bit apparent in their culture. In the end, however, as a team we were able to deliberate, discuss and strategies our way through these problems to achieve the stage one goal of deciding which project to undertake.
Entering into stage two of this project brought forth another host of both minor and major problems. Initially, once stage one was over, it was Chinese New Year so naturally there were a couple of meetings consisting simply of Scottish students, whilst the rest of the group enjoyed some time off. There was no language barrier or clash of culture to contend with during these meeting so everything went smoothly.
The way that they IGP was run from February until June, this enclosed the entire Scottish students university semester including exam period, for the Chinese students this enclosed the Chinese New Year and the first half of the semester. As previously noted the Chinese students missed a few weeks to celebrate, this wasn’t a big problem since development had not yet started up. However during the later weeks of stage 2, the project hit some bumps while the Scottish students were preoccupied. Progress was not as fast as it had been scheduled to be, this disappointed the Chinese students and they were very concerned that the project was going to fail. This was highlighted in the meeting minutes:
05/05/2015
Calum states that none of this is helpful unless they think the Scottish team is going to fail or there supervisor does
o Scottish team reassure that we are not going to fail
This resulted in a number of group emails going back and forth.
23/04/2015 Hi Kim,You said you have finished some functions but no web pages in last meeting. When I heard that, I am very afraid.
I beg your plan and deadline for each week again.
Thank you so much.Johan
Hi Johan,
I'd like to put you at ease and say that we work differently here, and we will 100% have the project finished. I understand your concern as we have not had as much progress on the site as neither you or I would have wanted.
We need to work together with a positive relationship and understanding of differences between ourselves. We understand your concern and that there is massive pressure on this project from both universities. I believe we will be successful, I hope you too can believe this?
Thanks
Patrick
To summarise these this involved the Scottish students reassuring the Chinese students that we would be able to make a success of this project and that it would be completed. It took a lot of persuading but eventually everyone was happy to trust in each other to complete their role within the group.
Stage 2 consisted mainly of two large problems, the first of these was the desire of the Chinese students to implement some form of rating system for users and groups. This was no surprise to the Scottish students, due to the previous insights into Chinese culture, they have a desire to quantify and measure everything. They also need to make sure people are following rules and not abusing the system, the feeling is this is also ingrained within Chinese upbringing, having slightly communist tendencies. As a reaction to these notions the Scottish students were immediately against this idea, even when it was brought up in the first meeting.
10/03/15
Evaluation
o Evaluation of students suggested by Ivy (5 star)
o 5 star rejected by UK team
o Group rating suggested – Similar to Facebook Like or
o StackOverflow
It took several weeks of deliberating during the meetings, many different systems were suggested and examples taken from other sites such as Facebook and LinkedIn.
17/03/15
Calum does not like rating system Max says we have a system of likes and kind of dislikes
o We think comments and rating on students is not a valid functiono We point out it is easy to implement
Nick suggests we have a like system for GROUPSo Do no dislikeo Ivy says you need a way to find out the contents for groups in order to
choose o Calum suggests a LinkedIn sort of functions were you can +1 skillso Group says like or nothingo Bill says ‘Rating system is indispensable\
Kim says she will draw up a diagram or schema as how it works o The Chinese side points out some of the groups only exist as a one off so
a group rating system may not work.o Max says you can still like the groups and it will be saved in the DB or
something along those lines
Kim points out that group can be liked and everyone involved will get rep. Nick points out the rating system is COULD HAVE and we shouldn’t stress Tristan likes Kim’s idea of everyone in the group getting rep
There was also trouble with the language and distance barrier at this point as the Scottish students struggled to get a grasp on exactly why the Chinese students wanted this feature so badly; and also not being face to face it made it much more difficult to express each other’s feelings to the group. Whenever it was felt that there was progress made regarding a rating system, the Chinese students found it very hard to disagree with people directly within a meeting, and so in the next meeting they would push their idea again. This was extremely frustrating for all members of the group as we felt that the precious time we had, with one meeting a week, was being wasted by going round in circles. This topic refused to go away throughout stage 2.
07/04/15
Talked more about rating systemo Going for ‘Linkedin’ demo
Eventually in the later weeks of the stage it was agreed that we would have a group based endorsement for each user in the system, where users can only endorse other users once per group. This appeared to be a fair compromise and the Chinese students were content with this. This was a very clear split between the two sides of the project, and tensions and tempers were frayed during the process of coming to an agreement. Exasperated by the fact that we felt we had agreed and yet the Chinese members kept pushing for more and the Scottish members were opposed to large amount of monitoring.
The second of the major problems faced was the layout and organisation of the Zhengzhou university, major, class structure. It was determined in stage 1 that the Napier students would be able to have a clear layout of how the university system operated by gaining a copy of the database which runs the system from Napier. In response the Chinese had hoped that it too would be able to acquire the data from Zhengzhou University or at least supply web scraped data or test data that would be enough to highlight how the university is operated. This was requested by the Scottish members of the group in almost every meeting.
10/03/15
o Chinese group to find out university department structure
At first the delay was caused by the Chinese leader being away working elsewhere. Once he got back we were informed that the data is unattainable from Zhengzhou, this was a problem as having the entire data would have been perfect but sometimes that isn’t possible when doing projects. The next best solution would have been having example data, however the data that was sent was all in Chinese and was only one class schedule.
24/03/15
This was obviously not enough to be able to design the system database around both systems, we had to agree to leave a large amount of flexibility within the system so that different university systems would be able to effectively use the system.
12/05/15
o We explain that it works for all universities as you can select your classes and your major separately
This was not final, however, the Scottish members kept pressing for better test data so that we could implement it within the set up and sign up to make the experience of signing up sleek and simple for each university. Continually the Chinese refused to supply adequate data, this went back and forth for some time, and we feel that the language barrier, cultural differences and distance apart helped attribute to this issue. It culminated in the Zhengzhou students accusing the Scottish students of building the system purely with Napier system in mind, upon explaining that we could not factor in the Zhengzhou system as the relevant data was not supplied, they claimed that it was
supplied. This made for a very heated meeting. To summarise, the Scottish students managed to convey that the current system was suitable and flexible enough to support both universities, and potentially many more. The Chinese students agreed that they had not supplied sufficient data but if we changed our system to allow only one class selected during the sign up phase then this would be sufficient to suit their system better.
12/05/15
o Johan explains that they have 10 classes instead of 3 modules in uko This is too labour some to have in the sign upo Can we change this to allow nullo We agree to allow 1
The problem was that the system made users select a minimum of classes during the sign up, this was too high for the Chinese classes as they often have 10 or more classes. This compromise suited both parties and brought the entire group back onto the same page.
Tied into the University data problem was the fact that the two sides of the product would be controlled and operated by a single API. Because there were slight differences in the universities the Chinese refused to use the API as they claimed that it was not suitable and there would not be able to use it as their system did not have sufficient fields to populate the requests.
12/05/15
Discussion on how with both using the same system we can use the same API/PHPfunctiono Tristian doesn’t understand how this can work with some values not set
It took a long time to convince the Chinese PHP team that in order to get around this issue you could simply supply empty variables in the fields and the requests would still work.
12/05/15
o We try to explain that he can set the missing values = “” so as to get passed the validation and use the existing function
o Tristian agrees to try this method eventually
The language barrier and the lack of being able to read whether somebody truly understood half way around the world was the biggest issue in this regard. Eventually, however, the penny dropped so to say and the Chinese team adopted the API and was able to move forward with development.
In previous points, we have touched on several contributing factors that hampered further the issues and problems we had during stage 2. One of the biggest was not being face to face, it is taken for granted how much body language and reaction can help an international group bridge the language barrier. One problem we faced was not knowing whether other sides of the group understood or accept opinions, in Scottish culture we are forward in telling others when we do not understand or agree, unfortunately this is not the case in Chinese culture. On skype, grainy images, patchy audio and inconsistent connection culminated in being unable to read each other body language. This was especially apparent during the discussions about the differing university system layout and the need for a rating system.
12/05/15
o Problems with skypeo Scottish team want to show off website
Implemented search, related groups and chato Further skype problems
Agree to double everything onto WhatsApp for reference and clarity
On one occasion the Skype connection was so bad that we had to abandon the meeting in this format and continue using WhatsApp.
WhatsApp was really useful for talking as it was instant messaging that was reliable and the Chinese students could quickly translate the text rather than having to understand what we were saying. It also allowed the IGP members to be in constant communication out with meetings, for example when a group member had to notify that they would be unable to attend meetings. The only slight problem faced by using WhatsApp was the time difference between Scotland and China, occasionally there would be an urgent issue but because the other part of the group was asleep it would not be resolved until sometime later. The time difference was not always constant, daylight savings was different in both countries so we had to manage both these changes when they were not always apparent to the other side. This did result in meetings having to be moved by an hour here or there at short notice once we realised that daylight savings had happened. There was a degree of understanding and patience shown by the entire group in dealing with these problems.
During the first week of May, the Chinese Students had a few days break from their university work. 2 members of the group decided to travel away from the university in order to visit family and friends during their time off. All of the group members were happy with this as everyone is entitled to some days rest and leisure. However upon arriving back from their trips they were called to see the Chinese group leader and informed that they had been removed from the group. Obviously this caused a great commotion within the group as nobody was happy about this outcome. The two members had informed the group of their intentions they had not done anything to deserve this treatment. This really unified the group and improved relations due to the fact that we felt aggrieved by the actions of John. The remaining members of the group pleaded with Karl and Bill, the two removed, to apologise to John the Chinese leader. After some convincing Karl agreed to apologise, this resulted in him being reinstated to the group.
05/05/15
Karl states that he is back doing his job again
Unfortunately, it became apparent that during the meeting with John where Bill had been removed from the group that Bill had been overly volatile throughout the situation and this did not sit well with John. Because of this rift between Bill and John he was unable to return to the group. In one sense the ability to pull together when this situation arose showed great team spirit, to go with that we lost a member which left a bitter taste in the mouth of the group.
The final issue experienced in stage 2 was a difference in teaching. The two sides of the group had been taught slightly different practices surrounding using composite keys within a database.
05/05/15
o Chinese believe id should be a varchar rather than int type o Nick explains that using a string type is bad practice because id = primary key has to
be a numbero Tristan wants to create a composite key that is created by several ids concatenated
together and integers will be added together not concatenatedo Scottish team do not agree – max states via whats app:
This resulted in a difference of opinion on whether some of the id fields within the database were actually needed, whether this was redundant data or not. Both parties urging each other to ask lecturers and professionals about the best practise in this situation.
05/05/15
o Chinese team state that they spoke to teacher about this and they confirmed this is what needs to be done to the database and that string is more stabilize
This issue was never properly resolved, the system had already been implemented in one way and it would have caused too much hassle to change it drastically in order to suit everyone. The system worked and having an extra field whether necessary or not wasn’t making a difference to that.
Upon beginning of stage 3, immediately the group ran into a problem! After the debacle surrounding group membership from stage 2, a new member of the Chinese team was added to the Group in order to fill the void caused by the removal of Bill from the group. The new member, Colin, posed several problems. Firstly the group had not built up a repertoire with him during the previous stages. He was not familiar with our system and our direction for the product and the language barrier was more of a problem. There was still a bad taste surrounding the Chinese teacher’s treatment of the members of the group, adding another one could have simply made matters worse. In response to this the group pulled together to make Colin feel welcome, help him to fit in and become a participating member of the group.
During the previous stages of the project time constraints had not been an issue at all, however as soon as stage 3 began we immediately had the additional pressure of having a deadline fast approaching. To go along with this we had to trust that the different parts of the project would be completed well and on time. To negotiate this problem the group was in constant communication, if anybody needed help, anyone else who was free would help them out in order to complete the project. This showed great teamwork as many members of the group that were not accustomed to working directly with other members of the group were able to interact and work well to get tasks completed.
The biggest issue the group, especially the Scottish half, was the difference in technology. The hardware we had at our disposal was outdated, although this wasn’t a major problem adjusting and getting used to the old operating systems was still an issue. More than the hardware was the language of the machines, although the language could be changed, when browsing the internet the results were still in Chinese. This made it extremely difficult to browse for solutions to problems and to check syntax using online resources. Which in turn slowed down the entire development process. This was not the only thing that slowed down the development process, the internet connection was unbearably slow! This was a major problem across all areas of development, deployment and testing. Having everyone in the computer lab working online at the same time proved to be a mitigated disaster. After a day of trying to work around this issue using mobile networks and tablets
to try to develop successfully it proved fruitless. To begin with we turned to offline development which meant that the strain on the network was only evident during deployment, debugging and testing. The funny thing about being a group is that we have to work together and that’s exactly what we did in order to combat this problem. The group employed a loose paired programming regime while debugging which meant that the strain on the network was reduced even further. This also cut down on human errors that developers make.
Stage 3 was started by the Chinese teacher reviewing our progress so far, in rounding this off he again returned to one of the oldest and longest running issues throughout this project, the evaluation/ rating system. The lecturer suggested allowing users of the system to review and evaluate the groups in order to assess the productiveness of them. After the vigorous deliberations and discussions, we had finally agreed on the user rating/endorsements, and agreed not to do the group rating. Having this pushed upon the group at this late stage was not what we needed, already being pushed for time, struggling with the language barrier and poor internet meant we felt like ignoring this request. Although when our backs are against the wall this group has shown time and time again it can meet any challenge. We had a brain storming session surrounding how to fairly assign a rating to groups, we felt that having other users decide the rating was unfair as humans cannot be depended on to be honest, or evaluate group without bias. To get around this we decided the rating to be based on how active and participatory the group was. To decide how to do this we each wrote up the best way; we then compared and colluded to come up with the best solution. This was calculated by evaluating 3 things, the percentage of attendance of meeting, percentage of participants in the chat and the amount of endorsements within the group converted to percentage and this gave the rating. This is the best example of our group blue sky thinking coming together to solve a problem that satisfied everyone.
Evaluation of the ProductCB From inception to finalisation, ‘Study with me’ went through many different phases of development. Our initial task was to create a system which allows access to study needs in an easy, understandable and quick fashion. Figure 20 demonstrates the complexity that arises from one user interaction path.
Figure 21. Use Case for system
As such, it would be difficult to pin point each point pertaining to the original specification outlined at the start. Hence, In order to proficiently evaluate the system which covered such a large scope, a step through guide of features has been created. Here we discuss the different functions set out by the client (as reviewed through MOSCOW procedures which can be found within the appendix).
As consulting and creating groups was the main focus of the project, the first instance shown when logged in is a list of groups the member is part of. These were split into 3 distinct collections;
Group Admin – Groups the user had created themselves Group Membership – Groups which the user was part of which were created by another
member Defunct Groups – Groups which had recently been closed or were no longer active
From here, the user can peruse groups they are part of at a glance and quickly move through all the information they require. This can all be see in the screenshot bellow:
Figure 22. Main page of Website
With a quick look the user can instantly see the different qualities of a group from the module they are most closely associated with to the tags in which a member can search through to find the group they are looking for. You can also search through your own groups using search feature to aid in finding the specified group you are looking for.
In order to help users identify which groups have been updated, amended or closed, a notification feature was also added. This alerts the user to changes and the number of alerts is displayed in the header of the website as you can see in figure 23. The Notifications tab goes into further detail and allows the user a more in depth look into specific groups and allows them to navigate to that part of the site.
Figure 23. Notifications Tab
Each notification type is displayed with a different glyph depending on the update. As well as this, notifications are saved for a maximum of three weeks and the time it was posted is displayed in the table and updates in real time without a need to refresh the page. More information is available in the notable features section pertaining to the notifications functions. Each group conforms to the same layout giving all relevant details pertaining to that set group. Members, meetings and a live Chat can be seen in figure 24:
Figure 24. Group View
This page is fairly self-explanatory, with the Chat function facilitating the need for communication between group members (detailed in MOSCOW in Appendix). The Meetings section shows what meetings are coming up at a glance and the meetings which have passed. We can also see basic details of the group on the left. The group rating system was a function that was highly sought out by the client and was pushed all the way through development and user testing. The Rating system proved to be a cause for concern throughout the whole process but was ultimately created formulaically and is documented in detail in the Rating system documentation included in Notable Features. The hearts beside the names are referred to as ‘Reputation’ and are also covered in the Group Rating feature.
From this page, we can note that most of our initial specifications were met, but booking rooms was at the top of our MOSCOW priorities. This was incorporated into the meetings section, as room availability would vary from week depending on School schedule.
Figure 25. Meeting Details
Each meeting displays key details including the room and amount of space still available. This changes depending on the amount of people attending the meeting. This can be seen below once attendance has been confirmed.
Figure 26. Meeting confirmed
Keeping track of space available in rooms and which rooms were free was a big requirement. With this design, users can see the amount of space left and whether choosing a larger room is worthwhile for their group.
Figure 27. Choosing a time and room for booking
Meetings which have been marked as attending and that are yet to happen, then appear in the Meetings tab. This allows users to view meetings and manage their time at a glance. As we can see in Figure 28 below, the previous booking in the example sheet here has been added to the meetings to come giving the agenda and duration for ease of management.
Figure 28. Upcoming meetings
Finally, the last page that was particularly high in the specification list was the profile page. In Figure 29 we can see a basic page showing the users details and groups they are part of. This page was
designed to give enough information to the visiting user to identify another member but keep information selective so as not to be intrusive.
Figure 29. User Page
From going through the step through guide, it becomes apparent that most of the preliminary goals have been achieved. Further Moscow features identified have been met in other sections of the report such as back copies (see Implementation – Server section) and minimal page loading (only two screens to traverse for all information).
The end of each development cycle, the two groups always convened to talk about features still to be added. This always allowed as to keep on top of required functionality and features of the product as conveyed in this section.
There were features outlined however that the group feels would have benefitted the project greatly and can be added in future versions and expansion. These included such things as:
Auto Generated Time tableo A feature left out due to time constraints
Alarms to indicate session is starting Cross referencing room booking with pre-existing University bookings A more enhanced, inclusive version of the background manager.
In the end, the System stands up to the initial specification very well. It has all the key elements needed to create a study group creation system only missing superfluous addition aiding in ease of use and experience. As is, the system has been deployed and is accessible at atux.co.uk where users can sign up and form groups. The Android app still needs refining but is also available to demo with almost full functionality and differing main functions. With a bit more time and better development conditions in China, the project would be up to complete release standards.
Project EvaluationKR The aim of the project was to produce a working version of an application to present to a panel of professionals at the end of stage three. The application had to be developed through hard work from all members of the team and hence, communication, comprising and an understanding of culture difference. Overall, the project was a success for both the universities and the students involved with two applications presented at the end of stage three.
Working internationally with two completely different social and academic cultures proved counterproductive at points during the project as a lot of time was spent gaining an understanding for why individuals would act certain ways. The most apparent difference was brought to attention by the involvement of the lecturers and their role as supervisors. On one hand, the supervisor of the Scottish students, Robert Ludwiniak was comfortable in allowing the students to work through the project themselves with the knowledge that he was there if needed. He also provided support on an emotional level by praising the students whenever they deserved it or listening to their worries when situations got stressful. On the other hand, the Chinese supervisor, John Lu took a completely different approach by applying a lot of pressure onto the team as a whole and enforcing certain requirements at inconvenient stages. It was understood that as the supervisor he had a lot of pressure placed upon him to ensure that the project was completed successfully but his actions implied a lack of confidence in the group and at times, this was demotivating. This form of teaching caused for many disruptions in the form of the following cycle: decisions were made, work was started based on those decisions and shortly after this the Chinese students would completely change causing the decision to be invalid. Due to this pressure placed on the Chinese students, it was more important to them that a product was produced that at least portrayed the intended idea. The result of this was the implemented android app being mostly hard coded meaning it was not in a ready state for integration with the rest of the functions when approaching the final stage.
Any individual reading this document would agree that there is little documentation produced from the Chinese students, this is not because they didn’t contribute but merely because they do not wish to share any of the documents worth referencing in this report. As this was the first time a project like this has occurred with the Chinese, they were not willing to share many pieces of their work
including source code and have stopped access to the server where files were stored. This is making it difficult for their contribution to be documented other than merely defining the android app and word of mouth from the Scottish students.
In addition to finishing the project successfully with a working product, relations between Zhengzhou University of Light and Edinburgh Napier have never been better with both institutes incredibly optimistic about working together in the future; it was undoubtedly a worthwhile experience for everyone involved.
ConclusionCB From a technical stand point and for a design perspective, the website stands string with its many layers of validation, error handling and user feedback. It was been made with care from the bottom up and never feeling error prone. The Android app is on its way but needs a redevelopment on its validation and security. As it stands, it’s not hard to imagine implementing the other MOSCOW features in the future however, developing the system into a framework would be the worthiest successor, and not only sticking with Universities but branching out communities and helping other govern bodies regulate and set up meetings.
KR To conclude this report, it should be stated that this project was the first of its kind and it didn’t just successfully produce a working product but was also an experience that many could learn from; not only those involved. The authors of this document have tried to convey both the technical aspects of the product and the inner workings of the project with as much accuracy as possible; in hope that it effectively portrays the whole project exactly as it was.
Overall, this process has resulted in a useful application, life experience for those involved and a positive future for more projects like this one.
Thanks Sections
International Group Project (IGP) would like to give thanks to all of those who contributed to their success but more specifically, the following individuals:
Robert Ludwiniak
John Wu
Andrew Cummings
Brian Davison
Bill Buchanan
Meng Li
- And most importantly our fellow team members from Zhengzhou.
Appendix 1.0
Johan(Project Manager)
Tristan(Communications
Manager)
Johan(Head developer)
Karl(Developer)
Colin(Developer)
Michael(Developer)
Tristan(Developer)
Tristan(System Admin)
Ivy(Designer)
Ant(Documentation)
Kim Reid (Project Manager)
Calum Beck (Communications
Manager)
Max Atkinson (Head Developer)
Nick Hunt (Developer)
Calum Beck (Developer)
Kim Reid (Developer)
Patrick Kelly (Developer)
Antoine Noel (System Admin)
Patrick Kelly (Design
Supervisor)
Calum Beck (Designer)
Nick Hunt (Design)
Appendix 1.1
Appendix 1.2
Appendix 1.3 Progress So Far
Kim -
Documentation:
Confirmed specification document Done/IterativeRole Hierarchy and Management Done/IterativePresentation for China To doScheduling Done/Iterative Meeting Agendas/Minutes Done WeeklyEvaluation Function Demo DoneProject Report To do
Testing:
User Testing Done/Iterative
Development:
Support Android Repair (Calum) To do
Extra :
Manage trello IterativeManage google drive Iterative
Calum -
Documentation:
Meeting Agendas/Minutes Done WeeklyRating System Demo Done Blog Entries Done
User Interface:Create Designs for Web DoneCreate options for logo Done/Iterative
Android App:
Interactive Android App Prototype DoneFix the functionality of the android app Currently working on
- Connect app to database To do - Remove hard coded data To do- Connect to PHP Slim To do- SHA2 Encryption To do
Testing:
User Testing Done/Iterative
Extra :
Train team to use Git Done
Nick -
Extras :
Prototype for web Done
Database :
Database Design DoneDatabase Implementation DoneDatabase Modifications Iterative
User Interface:
Home Page of Website DoneSign up Done
- Enter Personal Details Done- Validation of Input Done- Choose Modules Done
Log In DoneDashboard Page Done
- Edit Details Done- Suggested Group To do- Search Groups To do- My Groups To do- Notifications To do
Add Group Window To do
Back End Development:
Majority of the php functions *refer to php functions list document*
Documentation:
How to implement functions document Done
(for Chinese team)ERD Diagrams DoneOverall System Diagram Done
Antoine -
Server Implementation :
VPN/Firewall DoneGIT Server DoneSetting Up Back Up DoneTest Restore of Back Up DoneWeb/Application Server DoneDatabase Server DoneProduction Environment DoneWeekly Maintenance/Bug Fixes Iterative
Extra:
Teach team how to use GitLab - Done
Max
Back End Implementation:
Sourced real data for application DonePhp Functions *Refer to php functions list for done* / to doDebugging Done/Iterative
Front End Implementation:
Chinese Translation of UI Working onSupporting Nick & Patrick To do
Database:
Database Implementation - Done
Documentation:
Meeting Minutes - Done/Iterative
Testing:
User Testing Done/IterativeSystem Testing To Do
Patrick
Back End Implementation:
Php Functions - *Refer to php functions list*
Database:
Database Implementation DoneDatabase Maintenance Done/Iterative
Front End:
Html Prototype DoneCreate Group (Front End) Done - Needs ModifiedTerms and Conditions Page To doContact Page To doAbout Page To doTerms and Conditions To doPrivacy Policy To do
Documentation:
Meeting Minutes - Done/Iterative
Intensified Schedule
Proposed Schedule - 7 weeks until China
Week 1
6 PHP Functions
Basis of Website Interface Created
- Colour Scheme- Pages- Component Layout- Navigation
Week 2
7 PHP Functions
Complete Implementation of Website Interface
Week 3
Implement Functionality of Website
Implement AJAJ
Week 4
Exams – Parallel Systems
Week 5/6
Exams – Database
Testing of Website Functionality – Basic Working Version
Implementation of Additional Functions
Week 7
Testing of Website Functionality – Advanced Version
Recursive Tasks
Agenda, Minutes, Report Documentation, Presentation Preparation
Appendix 1.4
Appendix 2.0
Appendix 2.1
Appendix 3.0
MoSCoW Document
Stage 1
Must Have
Book available rooms
Webpage with varied formats to fit different phone types
Universally understandable webpage
Data backed up on server
Create an activity
o Join an activityo Input and display specific study subject
Create temporary and permanent group
Time slots for study
User registrations - profiles
Should Have
Display quantity of free seats
Retrieve students timetable
Decipher the rooms which are pre booked for standard classes
Widget for Android
Report users for inappropriate use
Join groups already in study
Public and private groups
Homepage with visible suggested groups
Notification and suggestion for other room if your booking is overridden by a teacher
Suggest another similar room if preferred room is not free
Alert user 5 minutes before booked time slot begins
Could Have
Evolutionary algorithm
Lecturers can override rooms booked by students
Chat to students in specific rooms through app
Evaluate other students strengths and weaknesses
Anonymous statistical data analysis to make best use of available classrooms
Data stored on database based on the reliability of the user
Alarm to alert user when study is finished
Feature to notify students about free seats
Small screens on room with quantity of available seats
Stage 2 MoSCoW Document
Requirements in bold have been completed at this point
Must Have
Book available rooms
Webpage with varied formats to fit different phone types
Universally understandable webpage
Data backed up on server
Create an group
o Join an groupo Create a meeting for groupo Input and display specific study subject
Create temporary and permanent group
Time slots for study
User registrations
User profiles
Web app Notifications
Search for groups
Ability to create and join groups
Should Have
Display quantity of free seats
Decipher the rooms which are pre booked for standard classes
Widget for Android
Report users for inappropriate use
Join groups already in study
Public and private groups
visible suggested groups
Notification and suggestion for other room if your booking is overridden by a teacher
Suggest another similar room if preferred room is not free
Alert user 5 minutes before booked time slot begins
Description of groups with tags
Search on multiple fields
Edit meeting
Edit group
Could Have
Evolutionary algorithm
Chat to students in specific groups
Evaluate other students strengths
Anonymous statistical data analysis to make best use of available classrooms
Data stored on database based on the reliability of the user
Alarm to alert user when study is finished
Prevent overbooking
Won’t have
Retrieve students timetable
Small screens on room with quantity of available seats
Feature to notify students about free seats
Lecturers can override rooms booked by students
Stage 3 MoSCoW Document
Requirements in bold have been completed at this point
Must Have
Book available rooms
Webpage with varied formats to fit different phone types
Universally understandable webpage
Data backed up on server
Create an group
o Join an groupo Create a meeting for groupo Input and display specific study subject
Create temporary and permanent group
Time slots for study
User registrations
User profiles
Web app Notifications
Search for groups
Ability to create and join groups
Should Have
Display quantity of free seats
Decipher the rooms which are pre booked for standard classes
Widget for Android
System is robust to deal with inappropriate use
Join groups already in study
Public and private groups
visible suggested groups
Suggest another similar room if preferred room is not free
Description of groups with tags
Search on multiple fields
Edit meeting
Edit group
Could Have
Chat to students in specific groups
Evaluate other students strengths
Anonymous statistical data analysis to make best use of available classrooms
Data stored on database based on the reliability of the user
Alarm to alert user when study is finished
Prevent overbooking
Alert user 5 minutes before booked time slot begins
Won’t have
Retrieve students timetable
Small screens on room with quantity of available seats
Feature to notify students about free seats
Lecturers can override rooms booked by students
Notification and suggestion for other room if your booking is overridden by a teacher
Evolutionary algorithm
Appendix 4.0 Technical Risks
Risk Summary Probability (L,M,H)
Impact (L,M,H)
Effect Recommendations
Server Failure
Server failure due to software failure, hardware failure or natural disasters.
Low High Loss of files and data.
Prevention Regular backups
Actions Restore/Replace server and use backups to restore data
Server Setup China laws on encryption may affect start up
Medium Low Effect completion time
Action Use server based in China or use server service that
meets the criteria of both universities
Data Loss
Accidental Medium Medium
Loss of data
Prevention Regular database back ups Developing code on local server and database before
implementation. Secure requests to database when coding.
Actions Use backups to restore data
Deliberate Low Medium
Data Access Access to data may be unavailable at times Low Low No access to data
Action Data for rooms, modules etc. will be stored on a central
database.
Data Integrity Data is subject to changing. High Low Incorrect Data on
databaseAction
Automated scripts to check API’s for any changes.
Data Permission Access to data Medium Low No access to Zhengzhou University data.
Actions Produce example data for Zhengzhou University.
Booking RoomsMay not have authority nor access to book rooms
High Low No access to existing systems
Actions A virtual booking system can be used to demonstrate
the feature.
Risk Summary Probability (L,M,H)
Impact (L,M,H)
Effect Recommendations
Loss of team member(s)
Personal reason, disputes Medium High
Effect time of completion and will be detrimental to group skills set
Prevention Solve any issues with a supervisor
Actions Reallocate team roles to utilise skills. Revise project specifications
Accident, Illness or death Low High Loss of crucial work or
information
Prevention Ensure all work is stored in a shared environment Passwords to server etc. must be known by minimum of
two peopleAction
Consult supervisor
International Communication
Time zone difference and social media restrictions may cause difficulties in communication
Medium Medium Effect time of completion.
Prevention Set up multiple communications channels such as Skype,
Trello and WhatsApp. Regular meetings will be scheduled around availability
and time zonesActions
Solve issues with supervisor
Not Skilled for a task
An extra task emerges that requires a skill set not known of team members
Low Low Effect time of completion.
Prevention All specification should be set out at the beginning
based on available time and skill set.Actions
Additional specification must be subject to feasible.
Uncompleted/late Tasks
Tasks that are uncompleted/late on regular basis.
Medium Medium Effect time of completion.
Prevention Rules must be agreed for late tasks. Notice must be
given for late tasks to allow for better management.Action
Monitor late completed task and manage appropriately
Appendix 5.0
Appendix 5.1
Appendix 6.0
Add User FunctionDescription: This function adds a new student to the database as well as recording what they study if any. This function can only accept a JSON object sent by HTTP POST. If sent by GET, PUT or DELETE it will give a 404 error. Look at the structure of the JSON object you must send. There is an English and Chinese version. This function will return a JSON object.
Chinese Characters can be sent as they are and need no HTML encoding as UTF-8 Unicode is set on the database and PHP headers.
Please read inline comments for more information.
GIT Location: scripts/addUser.php
Location: http://web.igp.noel.me.uk/scripts/student/register/
Method: HTTP POST
Parameters: JSON Object
JSON Object Format:
English Example Chinese Example
{"language":"EN","first_name":"John","last_name":"Smith","DOB":"03/04/2003","university":"1","email":"[email protected]","username":"john15","password":"n1c3pa$$w0rd","school":"1","studying": [ "19", "41", "63" ]
}
{"language":"CN","first_name":"鲁法案","DOB":"03/04/2003","university":"2","email":"[email protected]","username":"bill-lu","password":"n1c3pa$$w0rd","school":"9","studying": [ "666", "667", "668" ]
}
JSON Object Parameters
language “EN” English students “CN” Chinese students
Doesn’t require second name, sends out Chinese emailfirst_name First Name for “EN” students
Full Chinese name for “CN” studentslast_name Second Name for “EN” students
DOB Date of birth - format must be (dd/mm/yyyy)
university ID of the university - (1: Napier, 2: Zhenzou)
email Email
username Username/Nickname
password Users password
school School or department ID a user’s choses
studying Array of module/subject PK ID’s Must be array to work Optional
Function
Creates SALT and encrypts password with SHA2 Converts UK time to “Y-m-d H:i:s” format Adds student to `students` table in database Adds rows to `studying` table with module/subject PK and PK of student Creates activation link and stores in `hash_codes` table Sends activation email to provided email Sets PHP session cookie (logged in)
Returns - JSON Object + (PHP SESSION COOKIE)
Successfully Added Student {"result":"successful","studentID":"71"}A PHP Session cookie is set so user is logged in. [Android App will need to store this session cookie]
Failed - No Language Code provided {"result":"Failed","message":"No language set"} Failed – Invalid language code {"result":"Failed","message":"Invalid language code"} Failed – Validation {"result":"Failed","message":"Failed validation missing X"} Failed – Email in use {"result":"Failed","message":"Email already exists"} Failed – Username in use {"result":"Failed","message":"Username already exists"} Failed – Database Error {"result":"Failed","message":"Database Error"}
Appendix 7.0
Appendix 8.0 – Complete Minutes
Date 24/02/15
Attendance: All Scottish team members – Chinese New Year
Discussions:
Database scripting
Preliminary work before Chinese come back
A basic design to give idea of what the app should look like
Calum, Nick and Patrick meet up and talk web design
Cohesive experience for user
User and web look alike
Specification for design
Calum – design implementation for android
Max – restful API for website
Create a detailed list of task of progress weekly
Check Trello – outstanding and keep it updated
Go over all of project management stuff with Calum
Antoine create presentation for demonstration of GitLab
record and send to the Chinese – Tuesday at 1
Book library room j - 11 AM Tuesday
Additional timetable aspects – difficulties involved with room booking
Ask Bill for criteria of module – oracle
Risk analysis
Next meeting
Reviewed and hand in PID
Round up design meeting
Round up database meeting
Git presentation
Date Tuesday 10/03/15
Attendance: All Team Members
Discussions:
Introduction to New Team Members
o Michael – Internet
o Ainsley – e-commerce
o Ant
Evaluation
o Evaluation of students suggested by Ivy (5 star)
o 5 star rejected by UK team
o Group rating suggested – Similar to Facebook Like or
o StackOverflow
Database
o Bill working on ERD [unsure exactly]
o Chinese group to find out university department structure
Design
o Discussed Colours for Mobile App
o Flat Design suggested
o Chinese group to work on templates
Backend Design
o Nick, Tristan & Karl to work on PHP
Date Tuesday 17/03/15
Attendance: All Team Members
Discussions:
Started by talking about agendao Checked that everyone was added to the Trello account; Michael was addedo Ainsley has left left the Chinese group, Michael and Ant are still in group
Bill talks about his databaseo He has created a db with tables and has updated the ERD and has been
added to the servero We have evaluation of students cropping up againo Tristan has added his php development to their server
Max has said we will add version 1 of the website to GitLab very soon. Once uploaded everyone will be able to work on
o We have made more views and made a template for the website; people can now design once uploaded
Nick enquires as to the data for the China classrooms and departmentso Johan acknowledges and says he’ll send the data later
Back to the DB, nick points out the DB is different from ourso Bill says he has only added one table o Nick says our system is very similar and we can merge the dev for both of
them
ROB enters the room
Nick says he will talk to Bill later outside of the group chat regarding the DB Kim says, whilst talking to the Johan on the subject, that the Chinese side should be
implementing the Android app whilst our side does the Web appo Tristan asks for Clarity on this point o This is debated as making the Dev fastero Everyone agrees to this on WhatsApp
Max points out we have major coursework’s and Exams coming up and we will work solely on project in April
Ivy asks how we can keep designs similar if working on Android and Web separatelyo We respond saying we can use this time to show work as well as talk about
how to move forwardo Once a product is created, we will demonstrate every week what we have
added Max asks if their group can get example data from John on their Classroom. We
have secured data Nick emphasises that he will speak to Bill over Facebook Bill says they know how the classrooms are organised
o If John has the data they will send. o We clarify that we can use test data
Karl asks if we want a picture of a classroom to say basic layouto He will send examples of the university classrooms tomorrow (18/03/2015)
Calum does not like rating system Max says we have a system of likes and kind of dislikes
o We think comments and rating on students is not a valid functiono We point out it is easy to implement
Nick suggests we have a like system for GROUPSo Do no dislikeo Ivy says you need a way to find out the contents for groups in order to choose o Calum suggests a LinkedIn sort of functions were you can +1 skills
o Group says like or nothingo Bill says ‘Rating system is indispensable\
Kim says she will draw up a diagram or schema as how it works o The Chinese side points out some of the groups only exist as a one off so a
group rating system may not work.o Max says you can still like the groups and it will be saved in the DB or
something along those lines Kim points out that group can be liked and everyone involved will get rep. Nick points out the rating system is COULD HAVE and we shouldn’t stress Tristan likes Kim’s idea of everyone in the group getting rep Karl says they should collect likes to increase popularity and if group admin doesn’t
appear after booking, how do we combat this? Antoine says they should be reported if they do not appear and an admin should deal
with this Callum makes point about people cheating Nick says this system is for university students and not children. It is for helping with
Study Patrick makes sure that Bill understands before trying to explain to the others. Rob talks about how the attendance system is a bit flawed with people not needing to
go to classes in certain systems (this is only for Scottish side) Nick says maybe it is up to the group admin to kick people from group
o If someone is sick they can say if they are not coming Max again tries to push the rating system to be added at the end Chinese group talk a lot in Chinese about these points Tristan says we don’t need a 3 chance system (ie 3 strikes)
o The admin can choose if people can keep people ino Karl says if someone likes one group over others and he stops attending then
they will be taken off other groups automatically after a lack of attendance
o -> Tristan WhatsApp We move on to the next topic: version control Max says again we will have version 1 on GitLab Kim asked Karl to put his ideas on paper - He agrees Johan asks when can we add our first version of the website
o We say start of April will have a working version o We say we will send them the source code
Antoine labours the point that they MUST check their code is working before uploading to the GitServer which is synchronised with the Websever.
o This can cause major problems with the webservero Make sure the web server database is configured so it connects as it will be
configured to the local machine Tristan asks exactly how everything works
o Antoine explains all code is sent to the webserver automatically. o PHP can break webservers
Date Tuesday 24/03/15
Attendance: All Team Members
Discussions:
Run Through of Agenda
Johan has completed third phase for the schedule, this is on trello
Max: We have added basic views (pages) to the site using a javascript library called backbone.js.
We are going to start the server programming once we have our sample database up and running which should be tonight. When version 0.1 is ready we will upload it to GitLab.
Karl wants to decide about the rating system, need to make final decision
Colour scheme, favours dark blue and light blue, not so much green
Scottish team finished coursework this week, project will move forward quickly
Connection is bad
Must continue with audio only
Johan app is very good! Cool features, need to expand and decide what goes where
Chinese team wants website on GitLab – Scottish agree
Antoine provides link
Evaluation/Rating:
o Karl: like for groups and like for user
o Compromise between groups on initial feelings
o Tristian: I think we can use label to highlight user's strength skill, It's a useful comment
o What do we want to like?
o How do we rate the groups, temporary and permanent have different systems?
o Likes/rep only last specified amount of time, groups 1 year, person whole university career
o Student in database has table for rep
Nick and bill conversing about phpmyadmin
Next Steps
Database should be available tonight
Chinese to continue with app
Ivy, keep on with the Design
Scottish to continue with website
Reviewing the meeting process: Scottish only
o A lot of wasted time
o Went round in circles
o Have a chairman
o Minutes provided to everyone
o Video quality didn’t help
o Have the questions set out before the meeting
o Skype is not as good in comparison to talking in person
o With more to talk about there we will be more efficient
Date Tuesday 31/03/15
Attendance: All Team Members
Discussions:
Databaseo Antoine sent email about connection scripto If Chinese team need a change then notify Antoine/Nick/Maxo Antoine will set up DB account for Tristano Put up to date ERD on Trello - Nick/Maxo Bill to send his ERD in order to merge
PHP functiono Nick to put list of functions we need on trelloo Android and website will rely on same php functions
o Tristan/Karl/Nick to work on these and merge themo Chinese to upload PHP for login
UI Designo Ivy shows off app designo When designing we will try to follow similar design on websiteo Ivy to design logo’s
Next week we will haveo Updated function listo Logoo More functionality on site and app
Date Tuesday 07/04/15
Attendance: All Team Members except Antoine(working, prior notification) and Bill(he was Sick, contacted Johan to notify him)
Discussions: Johan shows the progress in the Android app
o Looking good with slider on left. o Asked to do validation for exit differently
Showed Nicks demo website. o Karl volunteered to Translate the website from English to Chinese
Talked more about rating systemo Going for ‘Linkedin’ demo
Talked about logoo Cat was nice but not included as we do not understand how it pertains to the
website Kim relays Antoine’s points about the server
o Maintenance stuff Karl says we should have both sites exclusively in the language you speak
o We try to say the brand should always remain the same for recognitiono Calum has volunteered to make a demo to show our idea for
Just said to continue what we were doing as progress has been good
Date Tuesday 14/04/15
Attendance: All Members except Antoine (working notified prior)
Discussions:
Scottish team tried to explain in more detail the type of data we needed:o List of moduleso Departments
Chinese Team say that the university has too many campuses and colleges to get the large amount of module.
Scottish Team say that you should be able to talk to someone at the university to get the data electronically in JSON format or just plain text.
Bill wishes to change the database and send the modifications to Nick. Johan says that the first and second name fields in the database should be combined due to
Chinese name being displayed differently.o Max suggests we just change the Chinese form to family name and first name but
stored in the same fields - Change the front end. o Chinese say that this is not convenient. o Team decide that the Chinese name which is in one part will be placed in the first
name field and the validation for second field will be modified. Chinese want to ensure that one email to one account only validation is present in
applications. o Max ensures the team that this has already been done and it will be pushed onto Git
later today. Application will allow log in via email address or username. Chinese question the validation of usernames and max ensures them again that a username
can only be used once for one person. Karl suggest that he creates the webpage for the new function ‘background management’
function that will be used for statistical datao Calum asks the Chinese team to elaborate more on the functionality of this new
function o The Chinese team say that this function will predict the future of the data and
display in charts etc.o Calum asks in another way, that when they activate their suggested php function,
what would it do?o Max informs the Chinese time that you can use sql to query the data to get the
information needed (Big Data)o Tristan says that they already have open source code and that we should let them to
do it. o Calum says that we trust them to implement and push to git and then we will
understand what they want better.o The Chinese confirm that they will show us when they have created and finished a
demo.
The Scottish ask how many functions the android app has. Johan confirms that they have done:
o Login - works with o Get group list
The Scottish team tell the Chinese the functions they have done:o Sign upo Log ino Log outo Create groupo Join Groupo Get group detailso Suggested groupso Get moduleso Add users to group
Tristan wishes to know how we can compare the php functions for web and android and choose which one to use
o Max says that we should all use the PHP branch on GitLab and see how we can combine them.
Ivy spoke to Nick and Calum about logo and they informed her that they were still doing preliminary sketches and sent images of this.
Calum talks about the property issues we have with Git and that he is going to teach the Scottish members how to use it properly.
By next week:
Chinese:o Join Group PHP Functiono Log out PHP Function o Background Management Function Demo o (Max tells Johan that we have already done log out and they can use this instead, if
they are capable of doing this - Tristan said that he will help johan with the php)o Browse Groups PHP Function o Search Groups PHP Function o Leave Group PHP Function etc
Date Tuesday 21/04/15
Attendance: All Members
Discussions:
Showed our website to Chinese students proving validation works for email and Name Tristan showed graphs populating from test data that can be applied to user tracking
o Talked about backend functions; still need to finish ratingo Kaikai said he will finish more functions on tracking for next week.
Asked about data for Zhengzhouo Tristan says that John cannot get data for their University o We suggest making test datao Clarified why we need different subject data
We talked about comments function (whiteboard function)/comments in groups Asked to integrate php functions into Android App
o Said they are finishedo We will review them next week.o This is good.
Date Tuesday 28/04/15
Attendance: All Members except Tristan (sick notified Johan)
Discussions:
Antoine speaks about the Servero We have attained a https certificate for the Websiteo You can now access the website using https and will be redirected if you put in a
straight link We are waiting for Nick to talk about our functions Johan has finished app so we are going to review
o The app is in an unfinished state with a few flourishes here and thereo Certain buttons need to be changed and the php file needs to be amendedo With no proper connectivity, it is hard to tell how well it will functiono It is not near completion
Talk about background management systemo Karl says we should have a way to manage Userso This should allow us to import data to another DBo They ask about how we should display our data in terms of chartso We say a pie chart should be on the front page with the amount of people enrolled
from the different institutions These are the functions they wish to look at having
o Group activity every dayo Personal skills and growtho The most active time every weeko Users growth for a weeko Group total daily activity and increase quantify every dayo Total number of userso Users to create, participate in and complete group activitieso App three functions respectively using an averageo The classroom utilizationo How many times a classroom is used every dayo The proportion of leisure time to participation time in activities by usero Most popular classroomo Classroom activities are commonly used
Nick talks about the new PHP functions he as implemented.o We have modified sign up to send email o This allows you to validate your accounto We can now change details, modifying account
They ask how many functions we have made. o they have not built any functions but used the ones we have uploaded but they have
not been implemented in the Android app Karl is going to translate more of the website and he is going to build background functions
o do not know what functions they want Talked about whether the website will be different for China and UK
o Said the website auto detects the language on the computer and we will have a manual change
Date Tuesday 05/05/15
Attendance: All Members
Discussions:
Johan sent Kim agenda Tristan reads out the first issue on the agenda Database:
o Chinese believe id should be a varchar rather than int type o Nick explains that using a string type is bad practice because id = primary key has to
be a numbero Tristan wants to create a composite key that is created by several ids concatenated
together and integers will be added together not concatenatedo Scottish team do not agree – max states via WhatsApp:o We cannot use composite keys because we will end up with redundant data
(requirement of normalisation)o Nick says that this lowers performanceo Antoine states that we finished the db design weeks ago and is too late to changeo Chinese team state that they spoke to teacher about this and they confirmed this is
what needs to be done to the database and that string is more stabilize o Nick is going to contact John
Next point – tagso Scottish team believe this is good idea and was going to be implemented anyways
Next point on agenda:o Issue about web app: Features being the sameo Patrick tells the team the following:o Scottish team wrote the function list and supplied this to china team, they also write
the code for the functions that the Chinese team could use but they did not do thiso The android app does not connect to the api
The Chinese team are told that they must use nicks documentation that is on trello so that the data is consistent.. any new functions nick will write.
o Calum asks which functions are needed for the android app that we do not use already for the website
o Johan said that web and app need different functions eg scheduleo Calum says that we need more data from the Chinese in order to do this – we have
repeatedly asked for this data and never been given it o Johan says that this is not in the functions listo Calum specifies that it’s not in the function list because you have not provided the
necessary data to be manipulated – we can only do it with ours for proof on concept but if the Chinese want it for their uni they must provide data in the form of a timetable and more than one so that we can sort the classes that are in different rooms
o Johan said that he can send the data latero Scottish team are not taking his word for it because this has been said numerous
times beforeo Calum specifically asks if they have the data now and they must let us know if they
don’t o Johan says that they can give us their own timetableso Calum reiterates that out of everything on the project, his favourite part and the
reason he agreed was for the schedulingo Johan says we can use virtual data
o Turns out that the data is just on the app and only shows how the app could look but Scottish team want a working app and hence, why they think it’s not finished.
It is stated again that using nicks php functions will help a lot with the implementation of the android app
o Johan sends list of functions and said that he must make sure you have these functions
o Calum states that these are not functions, these are just functionality of the website – we have php functions that do these.
The Chinese team are asking that if it has the same functionality isn’t that oko Scottish team strongly reiterate again that we all have to use the same php functions
– use the same back end for consistence Ivy wishes to use a like method for the groups – this is agreed by Scottish team
o Antoine gives example of desktop facebook and facebook appo Calum sympathises and states that he understands we are using a lot of English at a
fast pace and they must let us know if they do not understand Johan asked for everyone’s work
o Kim sent this to him at his request every week that he has askedo He said that there is no Kim and Calum in the documento Calum states that this was the wrong document – that was the functions listo Kim reiterates that we are most definitely in the documento Calum states that there are three people who do web technologies and these are
the people in the functions list – which is the php functions Johan looked at the document and then said he still needed the work list
o But Patrick asked for Johan’s work list and that he has not responded to Kim’s request for this every time she has replied to his requests
o Johan said he sent the work list for his work and he will send it for the rest of the group later
o Patrick states that Johan’s work list is not correct because he is claiming things are done which they are not
Calum states that none of this is helpful unless they think the Scottish team is going to fail or there supervisor does
o Scottish team reassure that we are not going to fail KaiKai states that he is back doing his job again
o Johan states that Kaikai will do the background management app Next Johan has issue with the meeting table and what it is used for
o Nick states that the meeting table is for the info related to the meeting. o Johan is happy with this
Scottish team inform the Chinese team that they are planning on having a meeting with John
Date Tuesday 12/05/15
Attendance: All Members except Nick and Antoine both working (Nick joins on WhatsApp)
Discussions:
Problems with skype Scottish team want to show off website
o Implemented search, related groups and chat Further skype problems
o Agree to double everything onto WhatsApp for reference and clarity
Review of appo During sign up users select major not classes this concerns Scottish teamo Use of display picture is questioned
Discussing hosting images on server and saving url in db Having users host online themselves Both of these have problems
o Will discuss later Nick has some talking points submitted prior
o Are they using and reading the PHP function documentation (if not why not)o Have they tested the php functions to see if they work? (We need to know now if
they do not work)o Do they have any data for classes at their campus
Tristian explains he cannot use api for sign up as we have different systemso No dob or usernameo API will fail if he does a post with less datao Johan explains that they have 10 classes instead of 3 modules in uko This is too labour some to have in the sign upo Can we change this to allow nullo We agree to allow 1
Johan asks us to implement a separate sign up for separate universities that would allow Chinese students to select major and their subjects would be chosen automatically
o We explain that this would be possible if we had the data about classes and majors from them
o Johan iterates that he has supplied us with thiso We dispute this abhorrently o Johan says that the data for his timetable should be enough for us to implement this
systemo We disagree that it is simply not enough to build a system for
The Chinese accuse us of building the website just to suit our universityo We explain that it works for all universities as you can select your classes and your
major separatelyo Agree on having a flexible systemo During sign up minimum of 1 module/classo Some majors have optional classes even more reason to keep existing system
Discussion on how with both using the same system we can use the same API/PHP functiono Tristian doesn’t understand how this can work with some values not seto We try to explain that he can set the missing values = “” so as to get passed the
validation and use the existing functiono Tristian agrees to try this method eventually
Discussion of tags implementationo We request explanation of how tag cloud workso It is populated from the offline db in Chinao Tristian will email explanation later
We move on to reiterate that we have to be clear about our implementation with each other so as not to waste time implementing different systems
Agree to compare website and app and come up with functions that can be merged
Date Tuesday 19/05/15
Attendance: All Members
Discussions:
Informal discussion regarding trip to china and preparation for trip No skype today as Napier is closed Ivy wants to Discuss logo
o Hand logo is suggestedo Also book logo suggestedo Finally a play on msn logo suggestedo What do we want logo to stand for, studying community love happinesso Books look bad as logoso Agree to finalise in china
Moving on to discuss implementation of PHP api on androido Johan informs us that it is going well
Discussing attaching tags to groupso Johan thinks 1 is enougho We explain that it is similar to having a descriptive title and that more would be
bettero We agree on 5 as a suitable amount
How to add tagso Text editor or add existing tagso Explanation of how tags are stored in dbo Talked about api functions for tags
Discuss notificationso Android won’t have time to implement
Background Manager
Appendix 9.0 – Communication Breakdown Emails
Thu 23/04/2015 09:24Hi Kim,You said you have finished some functions but no web pages in last meeting. When I heard that, I am very afraid. In general, If one wants to implement a website, he must do these steps. 1.Determine the theme website sketch outline2.The collection of material3.Build web frame structure4.Fill the website content5.The definition of the style of the website6.Realization of Web interactive function7.The final test releaseBut now, You got it wrong order, we only saw 2 pages! Besides, a good website must have a good management system. If not, How can we manage the website? How can we manage users? How can we get some useful information? How can we import information? We only have 4 weeks, we must hurry up. If we haven't a deadline for each phase, we are not able to fulfill the task . I beg your plan and deadline for each week.
About the database, we still have a lot of work to deal with. I have viewed the database, there is no foreign key in many tables, do you think it is normal? Maybe we have forgot these things1.There is a unique identifier should record table2.The database object should have the same prefix3.Id can't be Integer4.The table should avoid nullable columns...
About Android App, I have done most of the functions. For example, login, signup, create group, view group, search group, comment, upload head image, view my joined groups,view my created groups, add class schedule, modify course, add course, delete course, etc. I also more time to improve and modify. Don't worry about Android App, I can do it.
I beg your plan and deadline for each week again.
Thank you so much.Johan
Hi Johan,
I'd like to put you at ease and say that we work differently here, and we will 100% have the project finished. I understand your concern as we have not had as much progress on the site as neither you or I would have wanted.
We don't follow strict guidelines for projects, I personally have never ever saw that procedure for completing a website(my job outside of university is a web developer). The importance for us was put on the back end functions because we all agreed that you would use these functions also in the implementation of the android app. In order to help you we focused on these. I understand that if you are using your procedure then this is not how you would have done it. I can only apologize for that.
The database does need work, you are 100% right! However, on multiple occasions we asked for the data about your university(you ensured us that this was available, this was one of the founding principles for choosing this project) you have only this week come up with sufficient data so we can understand the structure of your university, this could have caused us to redesign the database, luckily It has not. We can proceed with finishing this now.
I am believing in you completely that you will have the android app done, have faith that we uphold our agreement to finish the website. I don't know the schedule as I am not involved in that side of the workings, Kim will get back to you with an update.
We need to work together with a positive relationship and understanding of differences between ourselves. We understand your concern and that there is massive pressure on this project from both universities. I believe we will be successful, I hope you too can believe this?
Thanks :)
Patrick
Appendix 10 – Sample Code
SLIM Router:
<?phperror_reporting(E_ALL);
require '../Slim/Slim.php';require 'prepend.php';require 'functions/email-template.php';require 'emailConf.php';include 'functions.php';include 'getModules.php';include 'getGroups.php';include 'getGroupInfo.php';include 'addUser.php';include 'login.php';include 'logout.php';include 'addGroup.php';include 'removeStudentFromGroup.php';include 'joinGroup.php';include 'activation.php';include 'notConfirmed.php';include 'updateStudent.php';include 'studentData.php';include 'updatemodule.php';include 'groupChat.php';include 'groupMeetings.php';include 'getNotifications.php';include 'createTag.php';include 'getTags.php';include 'getCounts.php';include 'endorseToggle.php';include 'getEndorsements.php';include 'ratingSystem.php';
\Slim\Slim::registerAutoloader();
$app = new Slim\Slim(); // start it up and declare our routes
$app->get('/activate/:activation', 'activation');$app->get('/notConfirmed/resend', 'nc_resendActivation');$app->get('/notConfirmed/change/:email', 'nc_changeEmail');$app->get('/notConfirmed/delete', 'nc_deleteAccount');
$app->get('/modules/:universityID', 'getModules');$app->get('/groups/', 'getGroups');$app->get('/groups/join/:groupID', 'joinGroup');$app->get('/groups/:groupID/info', 'getGroupInfo');
$app->post('/groups/action/remove', 'removeStudentFromGroup');$app->post('/groups/action/close' , 'adminCloseGroup');$app->post('/groups/action/reopen', 'adminOpenGroup');$app->get('/groups/:groupID/rating', 'calculateRating');
$app->post('/groups/add', 'addGroup');$app->post('/student/register/', 'addUser');$app->post('/student/login/','login');$app->get('/student/logout/','logOut');$app->get('/student/register/validate/email', function () use ($app) { validateEmail($app->request()->get('email'));});$app->get('/student/register/validate/username', function () use ($app) { validateUsername($app->request()->get('username'));});
// Routes for update Student Details$app->post('/student/change/email','updateEmail');$app->post('/student/change/username','updateUsername');$app->post('/student/change/password','updatePassword');$app->post('/student/change/personal','updatePersonal');$app->get('/student/getAllData','getAllStudentData');$app->post('/student/change/modules','updatemodule');$app->post('/student/resetPassword/set', function () use ($app) { include 'resetPassword.php'; resetPassword(); });$app->post('/student/resetPassword/request', function () use ($app) { include 'resetPassword.php'; sendResetPassword(); });
$app->get('/groups/messages/all/:groupID','getAllMessages');$app->get('/groups/messages/check/:groupID/:lastMessageID','checkNewMessages');$app->post('/groups/messages/add','addMessage');
$app->post('/groups/meetings/add','addMeeting');$app->post('/groups/edit', function () use ($app) { include 'editGroup.php'; editGroup(); });$app->post('/groups/action/userAdd', function () use ($app) { include 'addToGroup.php'; addStudentToGroup(); });$app->post('/groups/meetings/freeRooms','getFreeRooms');$app->get('/groups/meetings/get/:groupID','getMeetings');$app->get('/groups/meetings/attend/:meetingID','attendMeeting');$app->get('/groups/meetings/cancelAttend/:meetingID','cancelAttendMeeting');$app->post('/groups/meetings/change', function () use ($app) { include 'editMeeting.php'; editMeeting();});$app->post('/groups/meetings/delete', function () use ($app) { include 'editMeeting.php'; deleteMeeting();});$app->post('/groups/toggle/endorse', 'endorseToggle');$app->post('/groups/endorsements/:groupID', 'getEndorsements');
$app->get('/notifications','getNotifications');$app->post('/tags/new', 'newTag');$app->get('/tags/:Tag_categoryID', 'getTags');
$app->get('/counts', 'getCounts');
$app->run();?>
Student class:
<?phpclass Student{ // property declaration
public $userExists = false;public $userID;
public $firstName;
public $lastName;public $DOB;public $universityID;public $schoolID;public $universityName;public $schoolName;public $bio;public $email;public $username;public $active;public $language;
function __construct($ID = 0) { global $db; // database connection
$getUser = $db->prepare("SELECT *,b.name AS universityName,c.name AS schoolName FROM `students` a INNER JOIN `university` b ON a.universityID = b.universityID INNER JOIN `school` c ON a.schoolID = c.schoolID WHERE a.studentID = :ID");
$getUser->bindParam(":ID",$ID);$getUser->execute();($getUser->rowCount() == 1 ? $this->userExists = true : $this->userExists =
false );if($this->userExists){
$data = $getUser->fetch();$this->userID = $data['studentID'];$this->firstName = $data['first_name'];$this->lastName = $data['last_name'];$this->DOB = $data['DOB'];$this->universityID = $data['universityID'];$this->schoolID = $data['schoolID'];$this->bio = $data['bio'];$this->email = $data['email'];$this->username = $data['username'];$this->universityName = $data['universityName'];$this->schoolName = $data['schoolName'];$this->language = $data['type'];if($data['active'] == 1) { $this->active = true; } else { $this->active =
false; }}
}
public function userExists(){
return $this->userExists;}
function suggestedGroups(){
if($this->userExists){ /* Returns group data related to currently studying */
global $db; $suggestedArray = array();
foreach($this->currentlyStudying() AS $module) { $query = "SELECT *,(SELECT COUNT(*) FROM group_membership b WHERE b.studentID = :studentID AND b.groupID = a.groupID) AS joined FROM `groups` a INNER JOIN `module` c ON c.moduleID = a.moduleID WHERE a.moduleID = :moduleID HAVING `joined` = '0'"; $q = $db->prepare($query); $q->bindParam("studentID", $this->userID); $q->bindParam("moduleID", $module); $q->execute();
if($q->rowCount() == 1) { $suggestedData = $q->fetch(); $suggestedArray[] = $suggestedData; }
}
return $suggestedArray; } }
public function assignedGroups(){
/* Returns assigned group data from database ONLY if exists */if($this->userExists){
global $db;// Find Groups assigned to$query = "SELECT *
FROM `group_membership` aINNER JOIN `groups` b ON a.groupID = b.groupIDINNER JOIN `module` c ON c.moduleID = b.moduleIDWHERE a.studentID = ?";
$q = $db->prepare($query);$q->execute(array($this->userID));$groupData = $q->fetchAll();
for($index=0; $index< count($groupData); $index++){$groupData[$index]['tags'] = array();//Query for tags and put in array$groupTags = $db->prepare("SELECT `name_en` FROM `Tag_Group` d
INNER JOIN `Tag` e ON e.TagID = d.TagID WHERE d.groupID = :groupID"); $groupTags->bindParam(":groupID", $groupData[$index]['groupID']); $groupTags->execute();
$groupData[$index]['tags'] = $groupTags->fetchAll();}return $groupData;
}}
public function currentlyStudying(){
/* Returns modules currently studying ONLY if user exists */if($this->userExists){
global $db;$studying = array();
// Store modules that student studies$query = "SELECT `moduleID` FROM `studying` WHERE `studentID`
= ?";$q = $db->prepare($query);$q->execute(array($this->userID));
foreach($q->fetchAll() AS $s){
$studying[] = $s['moduleID'];}return $studying;
}
}
public function currentlyStudyingFull(){
/* Returns modules currently studying ONLY if user exists */if($this->userExists){
global $db;$studying = array();
// Store modules that student studies$query = "SELECT a.moduleID, b.module_code, b.module_name FROM
`studying` a INNER JOIN `module` b ON a.moduleID = b.moduleID WHERE a.studentID = ?";$q = $db->prepare($query);$q->execute(array($this->userID));$studying = array();foreach($q->fetchAll() AS $s){
$row = array(); $row["moduleID"] = $s['moduleID']; $row["moduleName"] = $s['module_name']; $row["moduleCode"] = $s['module_code'];
$studying[] = $row;}return $studying;
}}
public function changeEmail($email){
/* Changes users email if valid and not in use by another */$returnValue = false; // by default
if($this->userExists){
global $db;if (filter_var($email, FILTER_VALIDATE_EMAIL)){ $query = $db->prepare("SELECT * FROM `students` WHERE `email` =
:email"); $query->bindParam(':email',$email); $query->execute();
if($query->rowCount() == 0) { $update = $db->prepare("UPDATE `students` SET `email` = :email
WHERE `studentID` = :studentID"); $update->bindParam(':email',$email); $update->bindParam(':studentID',$this->userID); if($update->execute()) { $this->email = $email; $returnValue = true; } }
} } return $returnValue; }
}?>
Appendix 11 – Web User Manual
Study with MeUser Manual
Contents
Getting Started Start your own groupRegister an account 1 Add a group 9Activating your account 2 Add a meeting 9Login 3 Book a room 10Forgot Password 3 Edit a group 10Log Out 3 Edit members 10
Edit meetings 11Delete meetings 11
The Basics Close group 11Finding a group 4 Re-open groups 12Joining a group 4Attending a meeting 5Cancel attendance 5 Your ProfileUpcoming meetings 6 Edit personal details 12Group Chat 6 Edit email 13View member profile 7 Edit username 13Endorsing Others 7 Edit password 13Leaving a group 8 Edit modules 14Notifications 8
Getting Started
Register an account
1. Click on Sign Up on the homepage2. You will be required to provide the following
information: Step One
Full NameDate of birthUniversityEmailUsernamePassword
Step TwoSchool of your universityAt least 3 modules you study
Activating your Account
After completion you will be sent an activation email to the email you provided on sign up. In the meantime you will see the not active page.
These give options to Resend the activation email, Change the email you signed up with Delete your account.
When you receive your activation email, click the link and your account will be activated.
Login
1. Click on “Login” located at the top of the homepage.
2. A popup login box will appear –3. Use your username/email and password to login.4. You will be alerted if your login details fail.
Forgot Password
1. Click on Forgot password on the login box.2. Enter your email address in modal3. Check your email for reset password instructions
Log out
Click on the log out button, found in the top menu on all user pages.
The Basics
Finding a Group
1. Click on the ‘Search Groups’ tab on the dashboard.2. This will show suggested groups and search
groups.3. Type into the search bar to filter results.4. Click on a group you wish to join
Joining a group
1. Go to group page2. Click join group
You will need to join a group before you can attend meetings or take part in group chat. You can only join public groups
Attending a Meeting
1. On the group page2. Click on “more” next to the meeting date you wish
to attend.3. A modal box will appear with more information
about the meeting.4. Click on the green button title “Attend”
Cancel Attendance
1. On the group page2. Click on “more” next to the meeting date you wish
to cancel.3. Click the red button titled “Cancel attendance”
Upcoming Meetings
To view all your upcoming meeting which you are attending. Click on the Meetings tab on the dashboard.
Group Chat
1. On the group page2. Go to the group chat and type a message.
The messages are instant and do not require you to refresh the page.
View Group Member
You can view your group members profile by clicking on their name when in the group page.
Endorse a member
1. On the group page2. Click love heart next to member3. If heart is filled you have endorsed user4. Click again to remove endorsement
You can only endorse member who are in the same group as you.
Leaving a group
1. On the group page2. Click the red “Leave Group” button
Notifications
To view your notifications, click on the notifications tab on the dashboard. This will list notifications from a range of activities.
Start your own Group
Add a group
1. To add a new group click on “Add Group” button on the dashboard.
2. This will open a form for you to enter details. These are: Provide group name Provide group description Select a module Tag (optional) Specify if group is public or private Add Users (optional)
Add a Meeting
1. On group page2. Click on the “Add Meeting”3. This will open a form for you to enter details.
These are: Meeting name Agenda Meeting Date Meeting time (start and finish) Book Room (optional)
Book Room
You can only book rooms with a meeting. This can be done when adding a new meeting or editing a meeting.
Only available rooms will be shown for the date and time of your meeting. Each room will show the number of seats
Edit Group
1. Go to the group you want to edit2. Click on the edit button located next to the group
name on the group page.3. This will allow you to edit the group name and
description. Please note you can only edit groups you created.
Edit Members
1. Go to the group you want to edit2. Click on the “Edit Members” button on the group
page.3. This show a selection box which will allow you to
add new members and remove existing members.
Edit Meetings
1. Click on the “more” button next to the meeting you want to edit.
2. This will open up a modal window with information about the meeting.
3. Click on the “edit meeting” button, this will show a new modal window allowing you to edit the details
Delete Meetings
1. Click on the “more” button next to the meeting you want to delete.
2. This will open up a modal window with information about the meeting.
3. Click on the “delete meeting” button, you will be need to confirm you want to delete the meeting
Close Group
1. Got to the group you want to close2. Click on “close group”3. Confirm you want to close group
Re-open Group
1. Click on My Groups tab on the dashboard2. Go to defunct groups3. Click on the group you wish to re-open4. On the group page click the red button titled “re-
open group”5. Confirm you want to reopen
My Details
Edit Personal Details
1. Click on My Profile tab on the dashboard2. Go to personal details and click “Edit Personal
Details” button3. Click update to make changes
Edit Email
1. Click on My Profile tab on the dashboard2. Go to account details and click “Edit” next to email3. Click update to make changes
Edit Username
1. Click on My Profile tab on the dashboard2. Go to account details and click “Edit” next to
username3. Click update to make changes
Edit Password
1. Click on My Profile tab on the dashboard2. Go to account details and click “Edit” next to
password3. You will need to provide your current password4. Click update to make changes
Edit Modules
1. Click on My Profile tab on the dashboard2. Go to university details and click “Edit
school/modules”3. Select school and modules4. Click update to make changes
Appendix 12 – Android User Manual
Study with MeAndroid User Manual
ContentsGetting StartedRegister an account 1Activating your account 2Login 2Forgot Password 2Log Out 3
The BasicsFinding a group 3Joining a group 3Attending a meeting 4Group Chat 4View group members 4Find Free rooms 4Add to schedule 5Edit/ Delete schedule 5
Start your own groupAdd a group 5Add a meeting 6Book a room 6
Your ProfileEdit personal details 6Edit Password 7Edit image 7
Getting Started
Register an account
3. Click on Sign Up on start up4. You will be required to provide the
following information: Step One
UniversitySchoolMajorMatriculation Number
Step TwoFirst NameLast NameEmailPassword
Activating your Account
1
After completion you will be sent an activation email to the email you provided on sign up. In the meantime you will see the not active page.
These give options to Resend the activation email, Change the email you signed up with Delete your account.
When you receive your activation email, click the link and your account will be activated.
Login
5. Enter login details on start up
Forgot Password
4. Click on Forgot password on startup.5. Enter your email address in modal6. Check your email for reset password
instructions
Log out
1. Click on menu icon
2
2. Click Settings3. Click on the red log out button
The Basics
Finding a Group
5. Click on tag cloud from menu6. Select a tag or use the search bar7. This will display groups
Joining a group
3. Go to group page4. Click join group
3
Attending a Meeting
5. On the group page6. Click on meetings at top right corner7. Select the meeting you want to
attend8. Click Join at bottom of meeting page
Group Chat
3. On the group page4. Go to the comments and type a
message.
View Group Members
1. On group page2. Click companions
4
Find Free Rooms
1. Click on menu icon2. Click free rooms3. Select Campus and time
Add to Schedule
1. Click on menu icon2. Click schedule3. Click on a tile4. Enter information
Edit/Delete a Schedule
1. Click on schedule item2. Either Click modify or delete
5
Start your own Group
Add a group
3. Go to tag cloud4. Click on plus button at bottom right5. This will open a form for you to
enter details. These are: Provide group name Provide group description Tag
Add a Meeting
4. On group page5. Click on meetings6. Click create7. Enter details. These are:
Meeting name Agenda Meeting Date Book Room (optional)
Book Room
You can only book rooms with a meeting. This can be done when adding a new meeting or editing a meeting.
Only available rooms will be shown for the date and time of your meeting. Each room will show the number of seats
6
My Details
Edit Personal Details
4. Click on menu icon5. Click on settings6. Click account info7. Click edit
Edit Password
5. Click on menu icon6. Click on settings7. Click account info8. Click edit9. Click revise password
Edit Image
1. Click on menu icon2. Click on settings3. Click account info4. Click edit5. Click revise image
7
Table of Figures
Figure 1. System for project.................................................................................................................18Figure 2.Basic Initial design for website...............................................................................................18Figure 3. Site Map – Visual representation created by Max Atkinson (Stars represent Group Admin Actions)................................................................................................................................................19Figure 4. Initial website with responsive design..................................................................................20Figure 5. Final Android app UI and Website........................................................................................20Figure 6. Study with me Icon...............................................................................................................21Figure 7. User Validation on Signup form............................................................................................22Figure 8. Screenshot from http://www.netindex.com/download/2,88/China/ - retrieved 17/06/2015.............................................................................................................................................................22Figure 9. Animation Validation example..............................................................................................23Figure 10. Android Tag Cloud..............................................................................................................24Figure 11. Timetable widget with two classes.....................................................................................25Figure 12. Android Flow Chart.............................................................................................................26Figure 13. Overview of Amazon Web Services....................................................................................29Figure 14 - Student Class Code............................................................................................................49Figure 15 - Get/Post HTTP...................................................................................................................50Figure 16- Route..................................................................................................................................50Figure 17 - PHP Slim Sample Code.......................................................................................................51Figure 18 - addUser.php......................................................................................................................51Figure 19 Group rating system............................................................................................................57Figure 20. 1 Star Group rating example...............................................................................................57Figure 21. Use Case for system............................................................................................................70Figure 22. Main page of Website.........................................................................................................71Figure 23. Notifications Tab.................................................................................................................72Figure 24. Group View.........................................................................................................................73Figure 25. Meeting Details...................................................................................................................74Figure 26. Meeting confirmed.............................................................................................................75Figure 27. Choosing a time and room for booking...............................................................................76Figure 28. Upcoming meetings............................................................................................................76Figure 29. User Page............................................................................................................................77
8