qcat documentationqcat documentation, release 0.1.0 frontend installation 1.change to project folder...

QCAT DocumentationRelease 0.1.0

Lukas Vonlanthen (CDE)

Dec 04, 2020

Contents

1 Glossary 3

2 Setup 52.1 Installation and Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Configuration 93.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4 Development 434.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.2 Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.3 Data Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.4 Elasticsearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.5 Data flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.6 User Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.7 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.8 Translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.9 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.10 Django Snippets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.11 Grunt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5 Contributions 595.1 Translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6 API 616.1 API Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

7 Hosting 777.1 Hosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777.2 Latest changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787.3 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

8 Indices and tables 838.1 To Do’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Index 85

i

QCAT Documentation, Release 0.1.0

Contents:

Contents 1


2 Contents

CHAPTER 1

Glossary

User Group

User Groups A User Group within the context of QCAT is a group of user sharing the same permissions. See UserGroups for more information.

WOCAT WOCAT is the World Overview of Conservation Approaches and Technologies, an established globalnetwork of Soil and Water Conservation (SWC) specialists, dedicated to sustainable land management.

Please refer to http://www.wocat.net for more information.

3

http://www.wocat.net


4 Chapter 1. Glossary

CHAPTER 2

Setup

2.1 Installation and Setup

2.1.1 Requirements

The qcat application needs a whole range of server applications:

• Apache webserver

• Postgresql >= 9.4 with postgis extension

• uwsgi: application server for python web applications together

• elasticsearch: search engine based on Lucene

• memcached: Memory caching

Postgresql

Postgresql 9.4 or bigger is needed as JSONP is used. Postgis 2.1 or bigger is needed as well.

uwsgi

uwsgi is needed at least in version 2.x or bigger. In the following explanations we use uwsgi in emperor mode.Furthermore uwsgi needs the python3 plugin and apache needs the uwsgi-proxy module:

apt-get install uwsgi-emperor uwsgi-plugin-python3 libapache2-mod-proxy-uwsgi

Other required packages

sudo apt-get install python3-pip python3-psycopg2 libpg-dev python3-pil memcached→˓libmemcached-dev nodejs nodejs-legacy npm

5


wkhtmltopdf: a version with patched QT is required. Get the current stable version at http://wkhtmltopdf.org/downloads.html

cd /tmpwget <download_file>sudo tar xvf wkhtmltox-0.12.3_linux-generic-amd64.tar.xzsudo mv wkhtmltox/bin/wkhtmlto* /usr/bin

2.1.2 Installation

Backend installation

1. Creating user ‘qcat’:

sudo adduser qcat && addgroup qcat:qcat

2. Create the application folder:

sudo mkdir qcat-livechown qcat:qcat qcat-live

3. Install virtualenv:

sudo pip3 install virtualenv

4. Now change privileges to user ‘qcat’:

sudo -u qcat -icd qcat-live

5. Get the application from the github repository:

git clone https://github.com/CDE-UNIBE/qcat.git source

6. Create virtualenv and activate it:

virtualenv -p python3 virtualenvsource virtualenv/bin/activate

7. Install all requirements python requirements for the application:

cd sourcepip3 install -r requirements/production.txt

8. Now we are ready to install the system based stuff. This is done with fabric from another computer:

fab deploy_demo

9. Now we can try, if the Django applicatio is valid:

cd qcat-live/sourcepython3 manage.py validate

6 Chapter 2. Setup

http://wkhtmltopdf.org/downloads.html

http://wkhtmltopdf.org/downloads.html


Frontend installation

1. Change to project folder and use NPM to install Bower and Grunt:

cd qcat-live/sourcenpm install -g grunt-cli bower

2. Install the project dependencies:

npm install

3. Change to qcat user and let Bower collect the required libraries:

bower install

4. Use Grunt to build the static files:

grunt build

2.1. Installation and Setup 7


8 Chapter 2. Setup

CHAPTER 3

Configuration

3.1 Introduction

3.1.1 Settings

Please refer to the Application settings of QCAT.

3.1.2 Questionnaires

A questionnaire consists of several objects which themselves can have multiple child objects. This represents thehierarchy of the questionnaire.

The elements of a Questionnaire are:

1. Questionnaire: The top node of the questionnaire, represents the entire questionnaire.

2. Category: A (thematic) group, corresponds to a single step in the questionnaire form.

3. Subcategory: Another element for grouping inside the category.

4. Questiongroup: A group of questions which belong together.

5. Question: A single question consisting of a key and value(s).

6. Key: The key of a question.

7. Value: The value of a question.

8. Links: Links to other questionnaires.

Most of the elements of a questionnaire can have a configuration of their own, stored in their configuration fieldin the database. However, some of the elements allow overwriting part of their configuration in the Questionnaireconfiguration.

The following table indicates which objects allow configuration in the database and which can have their configurationdefined or overwritten in the Questionnaire configuration.

9


Configuration in DB In QuestionnaireCategory No YesSubcategory No YesQuestiongroup Yes YesQuestion No Yes (only in questionnaire configuration)Key Yes NoValue Yes No

Todo: Questionnaire configurations can inherit from another one. This needs further documentation.

3.2 Questionnaires

3.2.1 Questionnaire Configuration

Contains the configuration of an entire questionnaire. It allows to overwrite certain default configurations of its childelements.

The configuration of a Questionnaire is stored in a JSON format in the configuration field ofconfiguration.models.Configuration.

See also:

configuration.configuration.QuestionnaireConfiguration

Parent element: (None)Child element: Category Configuration

Format

The basic format of the configuration is as follows:

{# See class QuestionnaireSection for the format of sections."sections": [{

# See class QuestionnaireCategory for the format of categories."categories": [

{# The keyword of the category."keyword": "CAT_KEYWORD",

# See class QuestionnaireSubcategory for the format of subcategories."subcategories": [

{# The keyword of the subcategory."keyword": "SUBCAT_KEYWORD",

# See class QuestionnaireQuestiongroup for the format of# questiongroups."questiongroups": [

(continues on next page)

10 Chapter 3. Configuration


(continued from previous page)

{# The keyword of the questiongroup."keyword": "QUESTIONGROUP_KEYWORD",

# (optional)"template": "TEMPLATE_NAME",

# (optional)"min_num": 1,

# (optional)"max_num": 1,

# See class QuestionnaireQuestion for the format of questions."questions": [{# The key of the question."key": "KEY"

}]

}]

}]

}]

}]

}

See also:

For more information on the configuration of its child elements, please refer to their respective documentation:

• Category Configuration

• Subcategory Configuration

• Questiongroup Configuration

• Question Configuration

• Key Configuration

• Value Configuration

sections

A list of Section Configuration.

links

A dictionary with the configuration of the links which are possible from this questionnaire. Please note that onlybase_configurations should be linked.

3.2. Questionnaires 11


Example

The following is an arbitrary example of how a configuration could look like:

{"sections": [{

"keyword": "section_1","categories": [

{"keyword": "cat_0","subcategories": [

{"keyword": "subcat_0_1","questiongroups": [{"keyword": "qg_14","questions": [{"keyword": "key_19","in_list": true

}]

},{"keyword": "qg_15","questions": [{"keyword": "key_20"

}]

}]

}]

},{"keyword": "cat_1","subcategories": [

{"keyword": "subcat_1_1","questiongroups": [{"questions": [{"keyword": "key_1","in_list": true,"is_name": true

},{"keyword": "key_3","form_template": "inline_2","max_length": 50

}],"keyword": "qg_1"

},{





"questions": [{"keyword": "key_2","max_length": 50,"num_rows": 2

},{"keyword": "key_3"


}]

},{"keyword": "subcat_1_2","questiongroups": [{"questions": [{"keyword": "key_4"

},{"keyword": "key_11","questiongroup_conditions": [">0|sample_qg_22"

]}

],"keyword": "qg_3"

},{"keyword": "qg_22","questiongroup_condition": "sample_qg_22","questions": [{"keyword": "key_27","questiongroup_conditions": ["=='value_27_3'|sample_qg_23"

]}

]},{"keyword": "qg_23","questiongroup_condition": "sample_qg_23","questions": [{"keyword": "key_28"

}]

},{"keyword": "qg_29","detail_level": "sample_plus","questions": [{





"keyword": "key_37"},{"keyword": "key_38"

}]

}]

}]

},{

"keyword": "cat_2","subcategories": [

{"keyword": "subcat_2_1","questiongroups": [{"questions": [{"keyword": "key_13","questiongroup_conditions": ["=='value_13_5'|sample_qg_18"

]}

],"keyword": "qg_10"

},{"keyword": "qg_18","questiongroup_condition": "sample_qg_18","questions": [{"keyword": "key_24"

}]

}]

},{"keyword": "subcat_2_2a","subcategories": [{"keyword": "subcat_2_2b","questiongroups": [{"questions": [{"keyword": "key_12","view_template": "textinput"


}]

}]





},{"keyword": "subcat_2_3a","subcategories": [{"keyword": "subcat_2_3b","subcategories": [{"keyword": "subcat_2_3c","questiongroups": [{"keyword": "qg_19","questions": [{"keyword": "key_5","in_list": true

}]

}]

}]

}]

},{"keyword": "subcat_2_4","subcategories": [{"keyword": "subcat_2_4a","questiongroups": [{"keyword": "qg_20","questions": [{"keyword": "key_25"

}]

}]

},{"keyword": "subcat_2_4b","subcategories": [{"keyword": "subcat_2_4b1","subcategories": [{"keyword": "subcat_2_4b2","questiongroups": [{"keyword": "qg_21","questions": [{"keyword": "key_26"

}]





}]

}]

}]

}]

}]

},{


{"keyword": "subcat_3_1","questiongroups": [{"questions": [{"keyword": "key_7"


},{"questions": [{"keyword": "key_8"


},{"keyword": "qg_13","numbered": "inline","questions": [{"keyword": "key_17"


}]

}]

},{"keyword": "subcat_3_2","questiongroups": [{"keyword": "qg_7","numbered": "prefix","questions": [{"keyword": "key_9"

}(continues on next page)




]},{"questions": [{"keyword": "key_10"

}],"keyword": "qg_8","max_num": 3,"min_num": 2

}]

}]

}]

},{

"keyword": "section_2","categories": [{


{"keyword": "subcat_4_1","questiongroups": [{"questions": [{"keyword": "key_14","filter_options": {

"order": 1}


},{"questions": [{"keyword": "key_16","conditional": true

},{"keyword": "key_15","conditions": ["value_15_1|True|key_16"

]}

],"keyword": "qg_12"

}]

},{"keyword": "subcat_4_2",





"questiongroups": [{"keyword": "qg_16","questions": [{"keyword": "key_21","view_template": "textinput","questiongroup_conditions": [">1|questiongroup_17","<3|questiongroup_17"

]}

]},{"keyword": "qg_17","questiongroup_condition": "questiongroup_17","questions": [{"keyword": "key_22"


}]

}]

},{"keyword": "subcat_4_3","questiongroups": [{"keyword": "qg_24","view_template": "bars","questions": [{"keyword": "key_29"




}]

}]

}]

},{


{(continues on next page)




"keyword": "subcat_5_1","form_template": "table_1","view_template": "table_1","table_grouping": [["qg_25","qg_27"

],["qg_26","qg_28"

]],"questiongroups": [{"keyword": "qg_25","questions": [{"keyword": "key_33"


}]

},{"keyword": "qg_26","min_num": 3,"questions": [{"keyword": "key_35"


}]

},{"keyword": "qg_27","questions": [{"keyword": "key_33"


}]

},{"keyword": "qg_28","min_num": 3,"questions": [{"keyword": "key_35"






}]

}]

}]

}]

}],"links": [{

"keyword": "samplemulti"}

]}

3.2.2 Section Configuration

Contains the configuration of a section.

The configuration of a Section is only part of the Questionnaire Configuration.

See also:

configuration.configuration.QuestionnaireSection

Parent element: Questionnaire ConfigurationChild element: Category Configuration

Format


{# The keyword of the section."keyword": "SECTION_KEYWORD",

# (optional)"view_options": {# Default: "default""template": "TEMPLATE_NAME",

# Default: false"media_gallery": true

},

# A list of categories."categories": [{

# ...}

]}



See also:


• Category Configuration

Also refer to the Example of a Questionnaire configuration.

keyword

The keyword of the section.

view_options

(Optional). An optional object containing configuration options for the view representation of the section.

• template: An optional template name. Must be a valid file name with .html ending in folder templates/details/section/.

• media_gallery: An optional boolean indicating whether to include a media gallery at the top of the sectionor not. The gallery contains all images attached to the questionnaire.

categories

A list of Category Configuration.

3.2.3 Category Configuration

Contains the configuration of a category.

The configuration of a Category is only part of the Questionnaire Configuration.

See also:

configuration.configuration.QuestionnaireCategory

Parent element: Section ConfigurationChild element: Subcategory Configuration

Format


{# The keyword of the category."keyword": "CAT_KEYWORD",


# Default: false





"use_raw_data": true,

# Default: false"with_metadata": true,

# Default: {}"additional_data": {

"QUESTIONGROUP": ["KEY"]}

},

# A list of subcategories."subcategories": [{

# ...}

]}

See also:


• Subcategory Configuration


keyword

The keyword of the category.

view_options

(Optional). An optional object containing configuration options for the view representation of the category.

• template: An optional template name. Must be a valid file name with .html ending in folder templates/details/category/.

• use_raw_data: An optional boolean indicating whether to add the raw category data to the template or not.These values can then be used for example for manual rendering of the category details. Defaults to False.

See also:

configuration.configuration.QuestionnaireCategory.get_raw_category_data()

• with_metadata: An optional boolean indicating whether to add the metadata of the current Questionnaireto the template or not. Defaults to False.

• additional_data: An optional object containing the keywords of questiongroups and questions from whichadditional data is to be collected.

Example:

{"qg_1": ["key_1"]

}



form_options

• helptext_length (int): Overwrite the default length (number of words) of the helptext shown initially(without the “See more” button).

• numbering (str): An optional numbering of the category.

subcategories

A list of Subcategory Configuration.

3.2.4 Subcategory Configuration

Contains the configuration of a subcategory.

The configuration of a Subcategory is only part of the Questionnaire Configuration.

See also:

configuration.configuration.QuestionnaireSubcategory

Parent element: Category ConfigurationChild element: Questiongroup Configuration

Format


{# The keyword of the subcategory."keyword": "SUBCAT_KEYWORD",


# Default: false"raw_questions": true,

# Default: None"table_grouping": []

},

# (optional)"form_options": {# Default: "default""template": "TEMPLATE_NAME",

# Default: """label_tag": "h3",

# Default: """label_class": "top-margin",





# Default: []"questiongroup_conditions": [],

# Default: """questiongroup_conditions_template": ""

},

# A list of questiongroups."questiongroups": [# ...

],

# A list of subcategories."subcategories": [# ...

]}

See also:


• Questiongroup Configuration


keyword

The keyword of the category.

view_options

(Optional). An optional object containing configuration options for the view representation of the question.

• template: An optional template name. Must be a valid file name with .html ending in folder templates/details/subcategory/.

• raw_questions (bool): An optional boolean indicating whether to (additionally) render the questions intheir raw format.

• table_grouping: A nested array to define the layout of the table based on the keywords of the question-groups of the subcategory.

Example:

"table_grouping": [["qg_25", "qg_27"],["qg_26", "qg_28"]

]

Creates the following table (with "view_template": "table_1"):

qg_25 qg_26qg_26



form_options

(Optional). An optional object containing configuration options for the form representation of the question.

• template: An optional name of a template to be used for the rendering of the subcategory form. The nameof the template needs to match a file with the ending .html inside questionnaire/templates/form/subcategory/. If not specified, the default layout (default.html) is used.

Please note that some templates require additional options to be set.

• label_tag (str): Specifies the tag used for the label (eg. h3). Used only for templatehas_subcategories.

• label_class (str): Specifies an (additional) class name for the label tag. Currently only used for templatehas_subcategories.

• questiongroup_conditions (list): A list of questiongroup conditions to be passed to the subcategorytemplate in case of special rendering. Must correspond to the list of questiongroup_conditions set inthe form_options of the first key of the first questiongroup.

• questiongroup_conditions_template (str): Indicate a field template to be used for the rendering ofthe question which renders the conditional question (eg. checkbox_with_questiongroup). Must beused in combination with questiongroup_conditions. Template must exist in form/fields/{}.html.


• numbering (str): An optional numbering of the subcategory.

• has_links (bool): This is only required if a questiongroup within the subcategories is used to link to otherquestionnaires. Defaults to false.

See also:

Link Configuration

questiongroups

A list of Questiongroup Configuration.

Important: The options questiongroups and subcategories are exclusive, they should not be set both atthe same time.

subcategories

A list of Subcategory Configuration.

Important: The options questiongroups and subcategories are exclusive, they should not be set both atthe same time.



Form templates

Every subcategory should render a <fieldset> and its label as <legend>. Inside the fieldset, the questiongroupsare to be rendered.

For nested subcategories, use template has_subcategories.

Templates for subcategories are situated in the folder templates/form/subcategory/. They have access tothe following variables:

• formsets (list): A list of tuples containing the configuration and the Django FormFormSet objects of thequestiongroups ([({}, <FormFormSet>)]).

• config (dict): A dictionary containing the configuration of the subcategory. All of the form_optionsspecified in the configuration are available, as well as the following keys:

– form_template (str): The name of the template to be rendered next.

– has_changes (bool): A boolean indicating whether there are changes in this subcategory compared theolder version of the questionnaire.

– helptext (str): The helptext for the subcategory.

– label (str): The label of the subcategory.

– next_level (str): Indicates whether the next child to be rendered is another subcategory or a question-group. Possible values are subcategories or questiongroups.

– numbering (str): The numbering of the subcategory.

– table_grouping (from view_options)

– table_headers (from view_options)

– table_helptexts (from view_options)

– template (str): The name of the current subcategory template.

3.2.5 Questiongroup Configuration

Contains the configuration of a questiongroup.

The configuration of a Questiongroup is stored in a JSON format in the configuration field ofconfiguration.models.Questiongroup. It can also be overwritten in the Questionnaire Configuration.

See also:

configuration.configuration.QuestionnaireQuestiongroup

Parent element: Subcategory ConfigurationChild element: Question Configuration

Format


{# The keyword of the questiongroup."keyword": "QUESTIONGROUP_KEYWORD",






# Default: """conditional_question": "KEY_KEYWORD",

# Default: """layout": "before_table"

},


# Default: 1"min_num": 2,

# Default: 1"max_num: 3,

# Default: """numbered": "NUMBERED",

# Default: """detail_level": "DETAIL_LEVEL",

# Default: """questiongroup_condition": "CONDITION_NAME",

# Default: "" - can also be a list!"layout": "before_table",

# Default: """row_class": "no-top-margin".

# Default: "h4""label_tag": "h5",

# Default: """label_class": "",

# Default: """table_columns": 2

},

# A list of questions."questions": [# ...

]}

See also:




• Question Configuration


keyword

The keyword of the questiongroup.

Hint: Each keyword of a questiongroup needs to be unique throughout all questionnaires. This is because question-naire data is stored by their questiongroup keyword and when queried it needs to be mapped to the correct question-group.

view_options

(Optional). An optional object containing configuration options for the view representation of the questiongroup.

• template: An optional template name. Must be a valid file name with .html ending in folder templates/details/questiongroup/.

• conditional_question (str): For conditional questiongroups, the name of the key for which the ques-tiongroup will be rendered next to. Works for example with subcategory template “image_questiongroups”

• layout (str): Additional indications used for the layout. These depend largely on the template used. Knownvalues are “before_table” or “label”.

• raw_questions (bool): If set to true, raw questions are added to the template under the variableraw_questions.

• with_keys (bool): If set to true, a list with all the key labels of the questiongroup is added to the template(variable keys).

form_options


• template: An optional template name. Must be a valid file name with .html ending in foldertemplates/form/questiongroup/. If not specified, the default layout (default.html)is used.

• min_num: The minimum for repeating questiongroups to appear. Defaults to 1.

• max_num: The maximum for repeating questiongroups to appear. If larger than min_num, buttonsto add or remove questiongroups will be rendered in the form. Defaults to min_num.

• numbered: An optional parameter if the questiongroup is to be numbered. Currently, mainly thevalue display is used.

• detail_level: An optional parameter if the questiongroup contains additional, mostly more de-tailed questions which are only visible after clicking on a link. This is used for the “Plus”-Questions.The value of the parameter can be freely chosen.

• questiongroup_condition: An optional name of a condition valid for this questiongroup.The name must correspond to one of questiongroup_conditions of a Question configura-tion.

See also:



Question Configuration

• layout (str): General layout indications for the layout of the questiongroup inside the subcategory.This depends a lot on the subcategory template. Known values are for example “before_table”used in template “questionnaire/templates/form/subcategory/table_input.html” or “no_label_row”for tables.

• columns_custom (list): A nested list indicating the distribution of the columns, eg. [[“12”], [“8”,“4”]]. This is valid for template “columns_custom”.

• user_role (str): A specific configuration used only for template select_user.

• row_class (str): An additional CSS class for the <div class="row"> element containing allthe questions of the questiongroup. Example: “no-top-margin”.

• label_tag (str): Specifies the tag used for the label. Default is h4.

• label_class (str): Specifies an additional class name for the label tag.

• column_widths (list): Specify the column widths for the table. Used in template table. Ex-ample:

"column_widths": ["60%", "40%"]

• table_columns (int): Indicate the number of columns of the table. Used by templatetable_columns.

• helptext_length (int): Overwrite the default length (number of words) of the helptext showninitially (without the “See more” button).

• link (str): Required if the questiongroup is a link to other questionnaires. In this case, this valuemust contain the name of the configuration which is linked (eg. technologies).

questions

A list of Question Configuration.

Form templates

Templates for questiongroups are situated in the folder templates/form/questiongroup/. They have accessto the following variables:

• formset: A Django FormFormSet object, containing the (repeating) forms (formset.forms) as well asthe management form (formset.management_form) which needs to be rendered in order for the form tobe submitted correctly.

• config (dict): A dictionary containing the configuration of the questiongroup. All of the form_optionsspecified in the configuration are available, as well as the following keys:

– has_changes (bool): A boolean indicating whether there are changes in this questiongroup comparedthe older version of the questionnaire.

– helptext (str): The helptext for the questiongroup.

– keyword (str): The keyword of the questiongroup.

– label (str): The label of the questiongroup (if available).

– options (dict): The options of the keys, ({"key_1": {}}), to be passed to the template of thequestion.



– template (str): The name of the current questiongroup template.

– templates (dict): A dictionary of the templates of the questions ({"key_1": {}}), to be passed totheir templates

3.2.6 Question Configuration

Contains the configuration of a question.

The configuration of a Question is only part of the Questionnaire Configuration.

See also:

configuration.configuration.QuestionnaireQuestion

Parent element: Questiongroup ConfigurationChild element: Key Configuration

Format


{# The keyword of the key of the question."keyword": "KEY",


# Default: """label_position": "",

# Default: "h5""label_tag": "h5",

# Default: """layout": "stacked",

# Default: false"with_raw_values": true,

# Default: false"in_list": true,

# Default: false"is_name": true,

},


# Default: None





"max_length": 500,

# Default: 3"num_rows": 5,

# Default: """helptext_position": "tooltip",

# Default: """label_position": "placeholder",

# Default: false"conditional": true,

# Default: []"questiongroup_conditions": [],

},

# (optional)"filter_options": {

"order": 1}

}

See also:


• Key Configuration

• Value Configuration


keyword

The keyword of the key of this question.

view_options

(Optional). An optional object containing configuration options for the view representation of the question.

• template: An optional template name. Must be a valid file name with .html ending in folder templates/details/question/.

• label_position (str): An optional indication for the label placement. Possible values are: none (no labeldisplayed).

• label_tag (str): Specifies the HTML tag used for the label (eg. h3). By default, the tag <h5> is used.

• layout (str): Additional indications used for the layout of the question. Known values are stacked forstacked measure bars.

• with_raw_values (bool): Allows to also add the raw values (the keywords) to the list of values. This worksfor field types “checkbox”, “cb_bool” and “radio”. Defaults to false.

• in_list: An optional boolean indicating whether this question should appear in the list representation ofquestionnaires or not. Defaults to False, meaning that this question is not shown in the list.



• is_name: An optional boolean indicating whether this question represents the name of the entire Question-naire.

Important: Only one question of the entire Questionnaire can have this flag. If the key is inside a repeatingquestiongroup, only the first appearance of the key will be used as name.

• filter: An optional boolean indicating whether this question is filterable or not. If set to True, the questionwill appear in the filter dropdown.

form_options


• template: An optional template name. Must be a valid file name with .html ending in folder templates/form/question/.

• max_length: An optional integer to specify the maximum length of characters for this value. Renders as avalidator for text fields. This is only meaningful for key types char (default value: 200) and text (defaultvalue: 500).

• num_rows: An optional integer to define the number of rows to be shown for textarea fields. This is onlymeaningful for key type text. The default is 3.

• row_class (str): CSS class name added to the <div class="row single-item"> element containingboth the label and the field.

• label_columns_class (str): CSS class name added to the <div class="columns"> element con-taining the label.

• field_columns_class (str). CSS class name added to the <div class="columns"> element con-taining the field.

• helptext_position: An optional name for the placement of helptext related to the question. Possiblevalues are tooltip (showing the helptext as a tooltip on the question label)


• label_position: An optional name for the display and positioning of the label. Possible values are:placeholder (showing the label as a placeholder inside the input field)

• has_other (str): A name to be used by questions with key type “radio” to indicate that there is an addi-tional option “other” with a textfield to specify. The name must be unique and must be the same as used byother_radio of the other radio key.

• other_radio (str): A name to be used by the char key which acts as “other” radio button. Must be uniqueand match the name specified in has_other of the radio key and must be unique.

• field_options (dict): A dictionary containing options which are directly passed as attributes to the inputfield. This is currently only used for types int and float and allows for example to directly set the minimumor maximum values of the field.

Example:

{"field_options": {

"min": 1900,"max": "now"





}}

Please note that “now” is a special keyword and should only used with type int. This will be converted to thecurrent year.

For float values, there exists an additional field_option named decimals which is an integer specify-ing the number of decimal places in the output (details).

• question_conditions (list): An optional list of conditions triggering conditional questions within thesame questiongroup. Each condition must have the format "expresssion|condition_name" whereexpression is part of a valid (Python and Javascript!) boolean expression and condition_name is thename of a Question’s question_condition option.

• question_condition (str): The name of the condition to be triggered (as specified inquestion_conditions). Must be unique throughout the configuration.

• questiongroup_conditions: An optional list of conditions triggering conditional questiongroups. Eachcondition must have the format "expresssion|condition_name" where expression is part of avalid (Python and Javascript!) boolean expression and condition_name is the name of a Questiongroup’squestiongroup_condition option.

Example:

"questiongroup_conditions": [">1|questiongroup_17", "<3|questiongroup_17"]

See also:

Questiongroup Configuration

3.2.7 Key Configuration

The configuration of a Key is stored in a JSON format in the configuration field of configuration.models.Key.

Hint: The configuration of the key in the database cannot be overwritten in the questionnaire configuration. Thesame key can be used multiple times and in several questionnaires so any changes should be done carefully.

Format


{"type": "TYPE"

}

type

The following options are valid for type. Default type is char.



char A simple textfield for short text, with the label above the field.text A textarea for larger textbool A field for boolean values Yes (stored as True) and No (stored as False). Renders as radio buttons.measureA field to select measure values (e.g. low, medium, high). Also allows null values. Renders as button

group.The values of measure fields are stored as integers in the database. This allows easier queries such as“greater than ‘low’”.This type requires values related to the key to be present in the database.

checkboxA simple checkbox list to allow the selection of multiple values. Renders as checklist.This type requires values related to the key to be present in the database.

image_checkboxA checkbox list with images to allow the selection of multiple values. Renders as checklist with images.This type requires values related to the key to be present in the database and to have a valid image_nameconfiguration.

select_typeA select field which allows selection by typing. Renders the field with Chosen.select A simple select field without typing.cb_boolBasically a boolean field but rendered as a single checkbox (selected = true, not selected = false).date A datepicker field.file A field to upload any file (as opposed to imageimage A field to upload an image.hidden A hidden input field.radio A set of radio buttons, needs values.user_idA special field which allows to select a user.link_idA special field which allows to select another questionnaire.int A field rendered as input type=”number”, the validation makes sure it contains only integers.float A field rendered as input type=”number”, can contain decimal numbersselect_modelA select field similar to select_type which lets you select an instance of a model which has to be

specified in the form_options.display_onlyA field rendered as hidden in the form, but displayed like a textfield in the details.

3.2.8 Value Configuration

The configuration of a Value is stored in a JSON format in the configuration field of configuration.models.Value.

Hint: The configuration of the value in the database cannot be overwritten in the questionnaire configuration. Thesame value can be used in several questionnaires and for multiple keys so any changes should be done carefully.

Format


{# (optional)"image_name": ""

}


http://harvesthq.github.io/chosen/


image_name

The image_name defines the filename of the image associated with this value. This is in particular required forvalues belonging to a key with type image_checklist.

3.2.9 Link Configuration

Links to other questionnaires (eg. from Technologies to Approaches) can be configured within the questiongroup.However, some options need to be specified at subcategory level.

Example:

{"keyword": "SUBCAT_KEYWORD","form_options": {"has_links": true

},"questiongroups": [{

"keyword": "QUESTIONGROUP_KEYWORD__CONFIGURATION_OF_LINK","form_options": {

"template": "select_link","link": "CONFIGURATION_OF_LINK","max_num": 3

},"view_options": {

"template": "links"},"questions": [

{"keyword": "link_id"

}]

}]

}

Or more concrete:

{"keyword": "app__1__4","form_options": {"has_links": true

},"questiongroups": [{

"keyword": "app_qg__technologies","form_options": {

"template": "select_link","link": "technologies","max_num": 3

},"view_options": {

"template": "links"},"questions": [





{"keyword": "link_id"

}]

}]

}

Please note that the keyword of the questiongroup is crucial, it needs to contain the name of the configurationto be linked at the end, separated by __. Examples: app_qg__technologies linking to technologies ortech_qg__approaches linking to approaches.

For more information, please look at Subcategory Configuration or Questiongroup Configuration

3.3 Settings

3.3.1 Application settings

This chapter covers the configuration of the QCAT application. The pattern `config` from the 12 factor app iswidely followed, so many variables are stored in the environment.

For easy management of environment variables, the package envdir is used. The variables are stored in the folder`/envs`; your folder should look as follows

envs/AUTH_API_KEYAUTH_API_URLCACHE_URLDATABASE_URLDJANGO_ALLOWED_HOSTSDJANGO_CONFIGURATIONDJANGO_DEBUGDJANGO_SECRET_KEYDJANGO_SETTINGS_MODULEDJANGO_TEMPLATE_DEBUGDJANGO_USE_CACHINGMAINTENANCE_MODETESTING_FIREFOX_PATH

See also:

dj-database-url for the format of `DATABASE_URL`

See also:

django-cache-url for the format of `CACHE_URL`

Django settings

Most of the settings are identical to the Django settings. Please refer to the Django documentation for further infor-mation.

Following environment variables must be set in order for django to run:

• `DJANGO_CONFIGURATION` (for development: ‘DevDefaultSite’)


https://pypi.python.org/pypi/envdir

https://github.com/kennethreitz/dj-database-url

https://github.com/ghickman/django-cache-url

https://docs.djangoproject.com/en/1.8/ref/settings/


• `DJANGO_SETTINGS_MODULE` (for develpment: ‘apps.qcat.settings’)

• `DATABASE_URL`

• `DJANGO_SECRET_KEY`

QCAT settings

API_PAGE_SIZE

Page size of results for the API providing questionnaire details.

AUTH_API_KEY

AUTH_API_TOKEN

The API token used for the authentication.

Default: None

AUTH_API_URL

AUTH_API_USER

AUTH_COOKIE_NAME

AUTH_LOGIN_FORM

DEPLOY_TIMEOUT

Timeout between announcement of deploy and actual maintenance window in seconds.

ES_ANALYZERS

ES_HOST

ES_INDEX_PREFIX

ES_NESTED_FIELDS_LIMIT

ES_PORT

ES_QUERY_RESERVED_CHARS

Some charactes have special meaning to ES queries. https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-query-string-query.html#_reserved_characters

3.3. Settings 37

https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-query-string-query.html#_reserved_characters

https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-query-string-query.html#_reserved_characters


GOOGLE_MAPS_JAVASCRIPT_API_KEY

GOOGLE_WEBMASTER_TOOLS_KEY

GRAPPELLI_ADMIN_TITLE

GRAPPELLI_INDEX_DASHBOARD

HOST_STRING_DEMO

Used for continuous delivery (fabric).

HOST_STRING_DEV


HOST_STRING_LIVE


IS_ACTIVE_FEATURE_MODULE

Feature toggle for questionnaire-modules

IS_ACTIVE_FEATURE_SUMMARY

Feature toggle for summaries

IS_ACTIVE_FEATURE_WATERSHED

Feature toggle for questionnaire ‘watershed’

KEY_PREFIX

LOCALE_PATHS

LOGIN_URL

MAINTENANCE_LOCKFILE_PATH

MAINTENANCE_MODE

See https://github.com/shanx/django-maintenancemode


https://github.com/shanx/django-maintenancemode


NEXT_MAINTENANCE

See https://github.com/shanx/django-maintenancemode

NOSE_ARGS

PIWIK_API_VERSION

PIWIK_AUTH_TOKEN

PIWIK_SITE_ID

PIWIK_URL

REACTIVATE_WOCAT_ACCOUNT_URL

An URL to which users are redirected if the login failed because their account is not yet activated. Background is thatupon switching to the new WOCAT website in 2017, all existing user accounts have to be reactivated manually.

Default: https://beta.wocat.net/accounts/reactivate/

REST_FRAMEWORK

Settings for: http://www.django-rest-framework.org/

SEND_MAILS

SENTRY_DSN

See https://docs.sentry.io/clients/python/integrations/django/

SUMMARY_PDF_PATH

Path to folder to store/’cache’ created pdfs

SWAGGER_SETTINGS

See https://django-rest-swagger.readthedocs.io/en/latest/

TEMP_UNCCD_TEST

TESTING_FIREFOX_PATH

THUMBNAIL_ALIASES

3.3. Settings 39

https://github.com/shanx/django-maintenancemode

https://docs.sentry.io/clients/python/integrations/django/

https://django-rest-swagger.readthedocs.io/en/latest/


TOUCH_FILE_DEMO

Location of uwsgi-touchfile, used for continuous delivery.

TOUCH_FILE_DEV


TOUCH_FILE_LIVE


UPLOAD_IMAGE_THUMBNAIL_FORMATS

A dictionary specifying the different thumbnail formats for images. For every uploaded image, a thumbnail is createdin each of the formats.

Example:

UPLOAD_IMAGE_THUMBNAIL_FORMATS = {

# '[NAME]': ([WIDTH], [HEIGHT])'header': (900, 300),

'small': (200, 100),}

UPLOAD_MAX_FILE_SIZE

An integer indicating the maximum file size for a single file upload. In Bytes.

UPLOAD_VALID_FILES

A dictionary indicating what file types are valid for upload and with which extension they shall be saved.

Example:

UPLOAD_VALID_FILES = {

# 'TYPE': Used to group different types of files'image': (

# ('[CONTENT_TYPE]', 'FILE_EXTENSION')('image/jpeg', 'jpg'),('image/png', 'png'),('image/gif', 'gif'),

),'document': (

('application/pdf', 'pdf'),)

}



WARN_HEADER

Text to display as warn header at the bottom of the page.

WOCAT_IMPORT_DATABASE_URL

WORD_WRAP_LANGUAGES

List of languages to add the css-attribute: word-wrap. Use this with languages without spaces between words, such asKhmer.

3.3. Settings 41

CHAPTER 4

Development

4.1 Installation

This section covers the installation of QCAT for local development. For instructions on how to deploy QCAT to aserver, refer to the Deployment section.

4.1.1 Prerequisites

The following prerequisites are needed. Make sure to have them installed on your system.

Git

The source code of QCAT is managed in Git, a free and open source distributed version control system.

Quick Setup

Create a virtual environment and get the code:

mkvirtualenv --python=/usr/bin/python3.6 -a . qcatgit clone https://github.com/CDE-UNIBE/qcat.git .

Add the apps folder to your virtalenv:

add2virtualenv apps/

Switch to the branch you would like to work on:

git checkout develop

Set necessary env variables:

43

deployment.html

http://git-scm.com/


• envs/DJANGO_SETTINGS_MODULE

• envs/DJANGO_CONFIGURATION

• envs/DJANGO_SECRET_KEY

• envs/DATABASE_URL

Install requirements:: pip install -r requirements/development.txt

Run migrations and check if server is running:

python manage.py migratepython manage.py runserver

Build frontend stuff:

grunt build:deploy

For testing purposes, also set the following env variables:

• envs/TESTING_CHROMEDRIVER_PATH

• envs/NEXT_MAINTENANCE

• envs/GOOGLE_MAPS_JAVASCRIPT_API_KEY

Run the tests:

pytest

Docker

Use the provided docker-compose file to create the environment. Docker and docker-compose must be installed onyour system.

Preparation

Create a folder for the project:

$ mkdir qcat && cd qcat

Get the code:

$ git clone https://github.com/CDE-UNIBE/qcat.git .

Environment variables

Set following environment variables:

• DJANGO_SETTINGS_MODULE = qcat.settings

• DJANGO_CONFIGURATION = DevDefaultSite

• DJANGO_SECRET_KEY = random

44 Chapter 4. Development

https://docker.com/


Run

If you want to build all requirements (static assets, elasticsearch, database), pass the ‘build’ param.:

$ docker-compose run web build

If a database dump is available, copy it into the folder: /docker_compose/postgis/restore. Following file formats aresupported - keep in mind all files available are executed:

• .sh

• .sql

• .sql.gz

Open your browser and go to http://0.0.0.0:8000 to see the project up and running.

Hint: Docker

Refer to the official docs. Some useful commands to clean your system:

• docker system prune

• docker kill $(docker ps -q)

• docker rm $(docker ps -a -q)

• docker rmi $(docker images -q)

• docker volume ls -qf dangling=true | xargs -r docker volume rm

4.2 Data Model

4.2. Data Model 45

http://0.0.0.0:8000

https://docs.docker.com


4.3 Data Format

4.3.1 Questionnaire Storage

The data of questionnaires is stored in a JSON format in the data field of the questionnaire.models.Questionnaire model.

Questiongroups

• The data is stored grouped by questiongroups. Questiongroups are formed based on thematic coherence ofsimilar questions or if questions can be repeated multiple times.

• The keyword of configuration.models.Questiongroup (e.g. qg_1, qg_2, . . . ) is required as anidentifier of the questiongroup. This way it is not necessary to identify questiongroups based on their questions.

The basic format of the data is as follows:

{"qg_1": [{

# Keys and values of qg_1},{

# Repeated keys and values of qg_1},# ...

],"qg_2": [# (Repeated) Keys and values of qg_2

],# Further questiongroup data# ...

}

Keys and Values

Keys and values are in general stored as "key": "value" pair inside the questiongroup dictionary. However,some of the values are stored a bit differently, based on the type of the value.

• Predefined values: Predefined values (e.g. from dropdowns) are stored only as the keyword of theconfiguration.models.Value:

{"key": "value_1"

}

The translation of the value can then be looked up through the relation of the value to configuration.models.Translation.

• Multiple predefined values: Multiple predefined values (e.g. from checkboxes) are stored as an array ofkeywords of the configuration.models.Value:

{"key": [

"value_1",





"value_2"]

}

The translation of the values can then be looked up through the relation of the values to configuration.models.Translation.

• Freetext: Freetext values (e.g. from textfields) can be translated and are stored as dictionary with the locale inwhich they were entered as key:

{"key": {

"en": "Value 1","es": "Valor 1"

}}

This allows multiple translations for freetext. Each questionnaire has a many-to-many relation (questionnaire.models.QuestionnaireTranslation) to languages. This allows easy access tothe languages in which a questionnaire is available (without having to look inside the data JSON). This relationhas an additional field to flag the original language in which the questionnaire was filled out first.

Example

A concrete example of how the questionnaires are stored:

{# Specification of SLM Technology (Part 1)"tech_specification_1": [{

# 2.1 Definition of Technology (in one sentence)"tech_definition": {

"en": "Continuous breeding of earthworms in boxes for production of high→˓quality organic compost.",

"es": "Hacer una crianza permanente de lombrices para producir abono orgánico→˓de alta calidad."

},# 2.2 Description of the SLM Technology"tech_description": {

"en": "Vermiculture is a simple and cheap way to produce a continuous supply→˓of organic compost of high quality. Eisenia foetida, the Red Californian earthworm→˓(also called ‘the red wiggler’) is ideal for vermiculture since it is adapted to a→˓wide range of environmental conditions. Under culture, the worms are kept under→˓shade, in long wooden boxes filled with earth, cattle manure and an absorbent→˓material (eg straw). The box is covered by sheet metal (or wood, thick plastic→˓sheeting, or banana leaves) to protect the worms against UV radiation and birds/→˓chickens, and also to maintain a favourably humid microclimate. Fresh cattle manure→˓is a perfect food for the worms, but rotten coffee pulp can also be fed. Chopped→˓crop residues (eg cowpeas, leucaena leaves or other legumes) may be added.",

"es": "Se construye una canoa de madera en un lugar con sombra (ej. debajo de→˓árboles). La canoa se tapa con zinc, madera o otros materiales para protejer las→˓lombrices contra sus enemigos y para conservar la humedad. Un alimento adecuado→˓para las lombrices es el estiércol fresco de vaca, también se les puede echar→˓material vegetal verde (Terciopelo, Gandúl, Leucaena, etc.), preferiblemente picada→˓o semidescompuesta ya que las lombrices no tienen dientes más bien chupan→˓partículas de materia orgánica en el suelo. El abono se aplica como fertilizante a→˓cultivos, ha demostrado ser muy eficiente en sus efectos al aumentar la producción→˓de los cultivos. Es importante darle un mantenimiento continua a la lombricultura→˓(mantener humedad, dar alimento cada tres dias), de esta manera se produce abono→˓constantemente con cantidades cada vez más crecientes ya que las lombrices se→˓reproducen rápidamente en un ambiente adecuado. Se utiliza en la mayoría de los→˓casos la Lombriz Roja de California, la cual es un híbrido de varias especies,→˓criado en los años 50 en California para tener una lombriz prolífica, fácil a criar→˓en cautiverio y adaptado a diferentes medios. También se puede utilizar la Cubana→˓roja. Las lombrices son unos de los organismos principales en la cadena de la→˓descomposición de la materia orgánica y en la formación de humus estable en el→˓suelo."


4.3. Data Format 47



}}

],# Specification of SLM Technology (Part 2)"tech_specification_2": [{

# 2.4 Land use"cropland": 5, # High"forest_woodlands": 1, # Low"cropland_sub": [

"cropland_annual_cropping","cropland_rainfed"

]}

]}

Todo: This does not really belong here

Service output:

• With hierarchy (categories and subcategories)

4.4 Elasticsearch

Elasticsearch is used as a search engine to query the questionnaires.

Important: The indices in Elasticsearch only contain public Questionnaires ( with status = active). If pendingquestionnaires (eg. the pending changes of the current user) are to be displayed, use a database query instead ofElasticsearch.

4.4.1 Administration

There is an administration panel available for superusers. It allows to create and update indices. The panel can beaccessed in the user menu or directly via /search/admin/.

4.4.2 Structure

The different indices in QCAT are structured as follows:

/[index]/[type]/[id]

• index: [prefix][questionnaire_type] (examples: qcat_technologies,qcat_approaches).

– The prefix can be set in the settings.

– The index is actually an alias pointing to an index in behind (for example qcat_technologies_1).This allows to update the mapping of an index without any downtime (see also https://www.elastic.co/blog/changing-mapping-with-zero-downtime).


https://www.elastic.co/products/elasticsearch

https://www.elastic.co/blog/changing-mapping-with-zero-downtime

https://www.elastic.co/blog/changing-mapping-with-zero-downtime


• type: questionnaire. This is currently always set to questionnaire.

• id: identifier. The identifier of the questionnaire.

4.4.3 Document format

A document in elasticsearch contains the following entries (in _source):

{"data": {# A copy of the data dictionary as stored in the JSONB-Field of# questionnaire in the database.

},"list_data": {# A dictionary containing the entries used for the list# representation (as specified in the questionnaire configuration)

},"created": "[timestamp]","updated": "[timestamp]","code": "[code of the questionnaire]","name": {# A dictionary containing the translations of the name field (as# specified in the questionnaire configuration)

},"configurations": [# A list of the configurations in which this questionnaire is# available

],"translations": [# A list of the translations in which this questionnaire is# available

]}

4.4.4 Example

Example of a document stored in Elasticsearch:

{"_index": "qcat_test_technologies_1","_type": "questionnaire","_id": "101","_score": 1,"_source": {

"translations": ["fr","en"

],"data": {

"tech_qg_2": [{

"tech_definition": {"fr": "Ceci est la déscription 1 en français.","en": "This is the definition of the first WOCAT practice."

}


4.4. Elasticsearch 49



}],"qg_name": [

{"name": {

"fr": "WOCAT 1 en français","en": "WOCAT practice 1"

}}

]},"list_data": {

"name": {"fr": "WOCAT 1 en français","en": "WOCAT practice 1"

},"tech_definition": {

"fr": "Ceci est la déscription 1 en français.","en": "This is the definition of the first WOCAT practice."

}},"code": "tech_1","updated": "2015-02-10T16:07:20.847000+00:00","name": {

"en": "Unknown name"},"created": "2015-02-10T16:07:20.847000+00:00","configurations": [

"technologies"]

}}

4.4.5 Helper functions

The helper functions for Elasticsearch are inside the Django app search.

• search.search.advanced_search(): The main function used to query questionnaires based on theirconfigurations, codes, name and other attributes.

4.5 Data flow

• All data is saved in the Postgres database.

• Public questionnaires are put into the elasticsearch index.

• Converting a model instance into JSON (for elasticsearch) is handled by the serializer: questionnaire.serializers.QuestionnaireSerializer

• All relevant data (see below) must be available for objects from the model and elasticsearch; serializing anddeserializing must work in both ways.



4.5.1 Website

• The list view on qcat.wocat.net accesses eleasticsearch only.

• The detail view on qcat.wocat.net calls the method get_details on the configuration of the serialized ques-tionnaire.

4.5.2 API

• The list view lists all data from elasticsearch.

• The detail view on qcat.wocat.net calls the method get_details on the configuration of the serialized ques-tionnaire.

4.5.3 Serializing

• If possible, the serializer calls model properties.

• All data which is depending on the config calls a method on the serializer

• The configuration is loaded when the serializer is instantiated.

4.6 User Groups

The following user groups exist:

• Administrators

• Translators

4.6.1 Administrators

Access to admin page Yes

Administrators are basically superusers, having every permission there is.

4.6.2 Translators

Access to admin page No

4.7 Testing

4.7.1 Tests

QCAT is tested using functional tests covering the interaction on the web site and unit tests which test isolated piecesof the code.

4.6. User Groups 51


Important: The functional tests are based on Firefox. The Selenium WebDriver and its Python bindings used to runthe tests do not suppor the latest (default) version of Firefox.

Therefore, the functional tests are skipped by default. In order to run them, you need to first download an older versionof Firefox (e.g. version 28). Then you need to specify the path to this version of Firefox in your settings_local.py file, e.g.:

TESTING_FIREFOX_PATH = '/path/to/old/version/of/firefox'

Run all tests:

(env)$ python3 manage.py test

Run only selected based on app:

(env)$ python3 manage.py test accounts

Run only selected tests based on file path:

(env)$ python3 manage.py test functional_tests.test_login

Run only selected tests based on attribute:

(env)$ python3 manage.py test --attr=unit

(env)$ python3 manage.py test --attr='!functional'

The following attributes are available: unit, functional.

To disable the output of logging information, use the flag --nologcapture:

(env)$ python3 manage.py test --nologcapture

4.7.2 Coverage

You can measure the code coverage of your tests through Coverage.

Run the tests with coverage:

(env)$ python3 manage.py test --with-coverage

This will create a report which can be found under coverage_html/index.html.

4.7.3 Stress tests

To check the performance of all parts, some stress tests are included. Docker is required to run them, the tests arewritten with locustio. As locustio doesn’t work with python3, a dockerimage is provided. To execute the tests,proceed as follows (from the project root) and then open the browser:

(env)$ cd stress_tests && ./test.sh && cd -

Don’t forget to stop and remove all docker containers after the tests:


http://www.seleniumhq.org/

https://pypi.python.org/pypi/selenium

https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/

https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/

http://nedbatchelder.com/code/coverage/

https://www.docker.com/

http://locust.io/


$ docker stop $(docker ps -a -q) && docker rm $(docker ps -a -q)

To (force) remove all images, run:

$ docker rmi -f $(docker images -q)

To only create the dockerimage and run it, execute following commands:

(env)$ cd stress_tests(env)$ docker build -t locust .(env)$ docker run --rm -P locust

This starts a docker container with locust (on python 2.7). The IP address of the docker container is listed with:

$ docker network inspect bridge

With the IP for the running container, open the the URL <container ID>:8089. When actively developing the locustfile,the stress_tests folder can be linked into the docker image:

(env)$ docker run -v <path_to_qcat>/stress_tests:/locust -P locust --host=<your_host>

4.8 Translations

The source language of this project is English. Refer to the django docs for translations within django.

All fixtures for configurations and questionnaires are expected in English. Additional languages in the fixtures willnot be used.

4.8.1 Basics

• Translations are handled separately for the texts from django (i18n in python files) and questionnaires (i18n inmodels; see questionnaire.models.Translation).

• Historically, a jsonb field stores all translations. The structure is: {'configuration_type':{'keyword': {'en': 'string ahoy'}}}

• This is caused by the way configurations are created: as fixtures.

• Translation changes are expected to happen often, occasionally new languages will be introduced. Managingtranslations should require minimal effort from developers.

• These preconditions and the requirement for one system to handle all translations result in a workaround.

• gettext is the ‘default’ way to handle translations.

4.8.2 Workflow

• A string freeze is announced for before deploying to production.

• Creation of new .po files is therefore only allowed on the develop branch

• The command makemessages handles the following:

– Get latest resources from Transifex

4.8. Translations 53

https://docs.djangoproject.com/en/1.8/topics/i18n/translation/

https://wiki.openstack.org/wiki/StringFreeze


– If the -fp flag is set, the pull is ‘forced’. This means that local translation files are overwritten. By default,local files are not overwritten if they are newer than the version on Transifex.

– If new files are pulled, the po files are compiled (to mo files)

– Check, if new translations for configurations are available. If so, the content is written into a separatemodel. A temporary file based on this model is created, this temporary file is then parsed by djangos ownmakemessages command.

– Djangos makemessages creates .po files

– The user is asked if the new files should be uploaded to Transifex.

4.8.3 Recipe

• (evtl.: backup translations from Transifex): https://www.transifex.com/university-of-bern-cde/qcat/website/

• Activate virtual environment

• python manage.py makemessages -fp

• Release develop branch on master

4.8.4 Todo

• Check if txgh would be helpful.

• The current workflow is rather strict (no translations on feature branches, workaround with temporary file).Reviewing this workflow is recommended.

4.9 Deployment

While it is perfectly fine to deploy QCAT manually, it comes with a script which allows automatic deployment. Thisis especially helpful for developers to update the server with a single command.

The following section covers the automatic deployment with a Python library called Fabric.

4.9.1 Install Fabric

Fabric is not part of the requirements of the application and needs to be installed separately. At the time of writing,Fabric is not yet ported to Python 3.

(env)$ pip2 install fabric

If pip2 is not available, install fabric outside of the virtual environment.

4.9.2 Process

• Changes are always commited to feature branches.

• The git-flow model is in use.

• Only Selected people from the CDE (Centre for Development and Environment) can merge commits into thebranches develop and master.


https://www.transifex.com/university-of-bern-cde/qcat/website/

https://github.com/transifex/txgh

http://www.fabfile.org/

http://www.fabfile.org/

http://nvie.com/posts/a-successful-git-branching-model/


• Shippable is monitoring these branches, new commits trigger a deployment.

• The configuration for shippable is stored in the file: shippable.yml

– The host string is a secure value, which be generated on the platform.

– Before the actual deployment is started, all tests are run.

– If no tests fail, the deployment is started.

• Deployment is handled with Fabric.

• For the git branches develop and master, respective environments are created.

• Manual deployment should be avoided.

4.9.3 Provision

Warning: The following steps need to be done only once for each server but nothing should break ifyou run the command several repeatedly.

In order to prepare the server for deployment, you need to create a user with the appropriate rights on the host serverand create a folder for the app to live in. The script assumes the application will live in /srv/webapps/qcat/.Then run the following command:

(env)$ fab provision -H [user]@[server]

This will install the required software, create the folder structures for the app and get the latest source of the app fromits repository.

You will then have to create the environment variables and set the database connection and adapt other settings:Application settings

4.9.4 Deploy

To deploy the latest code to the server, use the following command:

(env)$ fab <environment> deploy -H [user]@[server]

If needed, add and enable the site in Apache.

4.9.5 Update recipes

Update server and reset database

1. Delete the database tables.

2. Run the update script:

[local] (env)$ fab deploy -H user@server

3. In Search Index Administration panel, delete all caches.

4. On the server, load the fixtures:

4.9. Deployment 55

http://www.shippable.com


[server] (env)$ python3 manage.py loaddata wocat.json

5. In Search Index Aministration panel: delete and recreate indices.

6. Eventually load data from Search Index Administration panel.

Update server

1. Run the update script:

[local] (env)$ fab deploy -H user@server

2. In Search Index Administration panel, delete all caches.

3. On the server, load the fixtures:

[server] (env)$ python3 manage.py loaddata wocat.json

4. In Search Index Aministration panel: delete, recreate and update indices.

4.10 Django Snippets

A collection of useful Django snippets.

4.10.1 Database migration

To detect the latest changes made to the data model and create a script reflecting these changes, use:

(env)$ python3 manage.py makemigrations

To apply the created migration script to a database, run:

(env)$ python3 manage.py migrate

4.10.2 Fixtures

Create fixture from database

Usage:

(env)$ python3 manage.py dumpdata [appname] --indent 4 > [file]

Example:

(env)$ python3 manage.py dumpdata configuration --indent 4 > sample.json

Use the flag -n if you don’t want to include primary keys in the fixture:

(env)$ python3 manage.py dumpdata -n --indent 4 auth.Group > accounts/fixtures/→˓initial_data.json



Load data from fixture

Usage:

(env)$ python3 manage.py loaddata [file]

Example:

(env)$ python3 manage.py loaddata sample.json

4.10.3 Graph Data Model

You can let Django (through django-extensions) create graphs of your data model automatically:

(env)$ python3 manage.py graph_models -a -g -o docs/images/qcat_datamodel.png

4.11 Grunt

GruntJS is a task runner which is used to compile and provide static files of QCAT. More information on Grunt can befound on their homepage.

4.11.1 Commands

The following grunt commands are available:

Command Actiongrunt Build (uncompressed) and run the servergrunt build Build (uncompressed)grunt build:deploy Build (compressed)grunt server Run the server

4.11. Grunt 57

https://github.com/django-extensions/django-extensions

http://gruntjs.com/

CHAPTER 5

Contributions

5.1 Translations

Thanks for wanting us to help us translate the SLM database! This article covers instructions to get you started.

5.1.1 Transifex

The translation of the strings on the Website is done with Transifex where you need a free account. We will then addyou as translators to QCAT on Transifex.

If you do not have an account for Transifex yet, you can send us your e-mail address and we will add you as a translatorto QCAT on Transifex. You will receive an e-mail with instructions to set up your Transifex account.

If you already have an account for Transifex, you can send us your username or the e-mail address of your accountand we will add you as a translator to QCAT on Transifex.

5.1.2 Translating on Transifex

Transifex has a very good documentation of the translation process on their website. Particularly helpful are the pageson Getting Started as a Translator and Translating with the Web Editor.

Please refer to their general documentation for further help.

5.1.3 Hints

No instant updates of translations

The translations on the QCAT website are not updated instantly. The developers need to trigger an update for the newtranslations to appear on the site. Therefore if you have finished your translations, let the developers know that theyshould update the website.

59

https://www.transifex.com/

https://www.transifex.com/university-of-bern-cde/qcat





https://docs.transifex.com/getting-started/translators

https://docs.transifex.com/translation/translating-with-the-web-editor

https://docs.transifex.com/

https://qcat.wocat.net


HTML tags in translations

Sometimes, there are HTML tags (such as <strong></strong> in the translation. These tags need to be addedexactly as they are in the translation as well. Make sure to close the tags: after each <strong>, there should be a</strong>. If you are not sure whether you have closed all tags, there is a service to check for closing HTML tags.

It can occur that there are strings inside the HTML tags (such as <div class="row">Content</div>). Thesestrings inside the HTML tag (in this case row) should not be translated.

Examples:

Text Translation text<strong>Type of measure</strong>: refer to 3.6 <strong>Type de mesures</strong>: se référez à la sec-

tion 3.6<div class=”row”>Refer to 3.6</div> <div class=”row”>Se référez à la section 3.6</div><div class=”row”><divclass=”medium-6 columns”><imgsrc=”/static/assets/img/smallmedium_QTKEN05_1.jpg”></div></div>

<div class=”row”><divclass=”medium-6 columns”><imgsrc=”/static/assets/img/smallmedium_QTKEN05_1.jpg”></div></div>

Placeholders in translations

Curly brackets ({}) serve as placeholders and should be added exactly as they are in the translation as well. Sometimes,they contain keywords (such as {user}) which should not be translated but added identically to the translation.

Other placeholders are %(keyword)s. Again, these need to be added exactly as they are in the translation.

Hint: In Transifex, these placeholders are highlighted in the original text in an orange color. You can click on thesehighlighted placeholders in the original text to copy them to the translation.

Text Translation textWelcome {} Bienvenue {}This questionnaire is locked for editing by {user}. Ce questionnaire est verrouillé pour modification par {user}.View this %(questionnaire_type)s. Voir cette %(questionnaire_type)s.

QCAT is currently under heavy development. If you are interested in the progress of the development, have a look atthe issues on the Redmine page of the project. Or see the List of ToDo’s for open issues.

60 Chapter 5. Contributions

https://www.aliciaramirez.com/closing-tags-checker/

https://redmine.cde.unibe.ch/projects/qcat

todos.html

CHAPTER 6

API

6.1 API Documentation

QCAT provides an API to query the contents of the Questionnaires.

The current version of the API is: v2.

6.1.1 QCAT API v2

Welcome to the documentation of version 2 of the QCAT API.

Additional information on the API can be found on the interactive documentation page allowing to try out availablerequests.

The example requests below are either for the command line tool curl or for the GUI application Postman.

• Authorization

• Available Configurations

– Example request with curl

• Available Editions for a Configuration


• Structure of a Configuration


• List of Questionnaires


– Filtering

61

https://qcat.wocat.net/api/docs

https://curl.haxx.se/

https://www.getpostman.com/


– Example request with Postman

– Code examples

• Details of a Questionnaire


– Example request with Postman

• Obtain Application Token


• Create a New Questionnaire


• Upload an Image


• Fetch Questionnaire Data


• Edit Existing Questionnaire


• List of User’s Questionnaires


Authorization

Requests to the QCAT API require an authorization token in the header of the request.

To access the following API endpoints, please contact the Wocat secretariat stating your activated Wocat account(required) and the purpose of your API requests. An authentication token will then be created for your account. Thistoken is denoted as AUTH_TOKEN in this document.

• Available Configurations

• Available Editions for a Configuration

• Structure of a Configuration

• List of Questionnaires

• Details of a Questionnaire

To access the following API endpoints, use the Obtain Application Token endpoint. This token is denoted asAPP_TOKEN in this document.

• Create a New Questionnaire

• Upload an Image

• Fetch Questionnaire Data

• Edit Existing Questionnaire

• List of User’s Questionnaires

62 Chapter 6. API


Available Configurations

Return the available Configurations in QCAT.

Endpoint: /api/v2/configuration/

Allowed method: GET

Request headers: Authorization: Your authorization token for the QCAT API. Specified in the format TokenAUTH_TOKEN

Accept: Optionally specify the format of the response. Available formats are JSON (application/json),XML (application/xml) and CSV (text/csv). Default: application/json

Optional Get parameters: flat: If present, the structure will be a flat list of questions.

Example request with curl

Request:

curl -X GET -H 'Authorization: Token 0e03f8061c83e58d1595c3b21ddce9712233cb4e' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/configuration/

Response:

{"configurations": [

["approaches"

],[

"cca"],[

"technologies"],[

"unccd"],[

"watershed"],[

"wocat"]

]}

Hint: If the response of your request is in binary format (e.g. weird characters shown on the screen), add theparameter –compressed to the curl command.

Available Editions for a Configuration

Return the available Editions for a Configuration in QCAT.

Endpoint: /api/v2/configuration/{code}/

6.1. API Documentation 63


Allowed method: GET





Request:

curl -X GET -H 'Authorization: Token 0e03f8061c83e58d1595c3b21ddce9712233cb4e' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/configuration/→˓technologies

Response:

{"editions": [

["2015"

],[

"2018"]

]}


Structure of a Configuration

Return the structure of the configuration of a questionnaire. Contains information about the categories, questiongroupsand questions that build a questionnaire.

Endpoint: /api/v2/configuration/{code}/{edition}/

Allowed method: GET





Request:

64 Chapter 6. API


curl -X GET -H 'Authorization: Token 0e03f8061c83e58d1595c3b21ddce9712233cb4e' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/configuration/→˓technologies/2018/

Response:

[{"categories_label": "1. General information","categories_keyword": "tech__1","subcategories": [

{"subcategories_label": "Image","subcategories_keyword": "tech__0__1","questiongroups": [{

"questiongroups_repeating": null,"questiongroups_label": "","questiongroups_keyword": "qg_image","questions": [{"questions_label": "Image","questions_keyword": "image","questions_type": "image","questions_choices": []

},{"questions_label": "Caption, explanation of photo","questions_keyword": "image_caption","questions_type": "char","questions_choices": []

},// ...

]}

]},// ...

]},// ...

]


List of Questionnaires

Return a list of Questionnaires. The same filter parameters as for the list view (search or filter) in QCAT can be passed.

Endpoint: /api/v2/questionnaires/

Allowed method: GET

Filters: Pass as query string, see filtering.





Optional Get parameters: Basically, the same parameters as for the list view in QCAT can be provided, namely page,type and all the filter parameters used for the list view.


Request:

curl -X GET -H 'Authorization: Token 0e03f8061c83e58d1595c3b21ddce9712233cb4e' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/questionnaires/

Response:

{"count": 1297,"next": "https://qcat.wocat.net/en/api/v2/questionnaires/?page=2","previous": "","results": [{

"name": "Participatory action research on drip irrigation","updated": "2017-06-28T14:43:10.622384Z","code": "approaches_2350","url": "/en/wocat/approaches/view/approaches_2350/","details": "/en/api/v2/questionnaires/approaches_2350/"

},{

"name": "Farmer field schools on integrated plant nutrient systems","updated": "2017-06-28T14:33:18.215042Z","code": "approaches_2351","url": "/en/wocat/approaches/view/approaches_2351/","details": "/en/api/v2/questionnaires/approaches_2351/"

},// ...

]}


Filtering

Filters can be passed as query string to the GET request. As an example, ?type=technologies will list technolo-gies only (full URL: https://qcat.wocat.net/en/api/v2/questionnaires/?type=technologies).

Hint: The query string for filtering on the API is the same as the one used on the default QCAT search (https://qcat.wocat.net/en/wocat/list/). So the easiest way to combine filters is to use the default search, and copy its querystring to your API call.

66 Chapter 6. API

https://en.wikipedia.org/wiki/Query_string

https://qcat.wocat.net/en/api/v2/questionnaires/?type=technologies

https://qcat.wocat.net/en/wocat/list/

https://qcat.wocat.net/en/wocat/list/


Hint: An additional filter (not available for the default QCAT search is the possibility to filterby edition, which limits the result to questionnaires available in a certain edition of a type (e.g.type=technologies&edition=2015).

Example URLs using the filters:

• Get all Technologies:

https://qcat.wocat.net/en/api/v2/questionnaires/?type=technologies

• Get all Approaches in Nepal:

https://qcat.wocat.net/en/api/v2/questionnaires/?type=approaches&filter__qg_→˓location__country=country_NPL

• Get all Technologies in altitudes from 3‘000 to 4‘000 m a.s.l. and with annual rainfall < 250 mm.:

https://qcat.wocat.net/en/api/v2/questionnaires/?type=technologies&filter__tech_→˓qg_56__tech_altitudinalzone=tech_altitudinalzone_3000_4000&filter__tech_qg_54__→˓tech_rainfall=tech_rainfall_less_250

• Get only Technologies which are available in edition 2015:

https://qcat.wocat.net/en/api/v2/questionnaires/?type=technologies&edition=2015



Example request with Postman

Code examples

• Example code to collect all Questionnaires (in Python, using the Requests library)

• Script library for download

Details of a Questionnaire

Return a single Questionnaire by its code. The returned data contains the full configuration (including labels ofsections, questiongroups etc.).

Endpoint: /api/v2/questionnaires/{identifier}/

Allowed method: GET



68 Chapter 6. API

https://gist.github.com/lvonlanthen/a11eb62838af3a294fc54e74d380a1ff

http://docs.python-requests.org/en/master/

https://github.com/CDE-UNIBE/qcat-api-scripts



Request:

curl -X GET -H 'Authorization: Token 0e03f8061c83e58d1595c3b21ddce9712233cb4e' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/questionnaires/→˓approaches_2350/

Response:

{"section_general_information": {"label": "General Information","children": {

"app__1": {"label": "General information","children": {"app__0__1": {

"label": "Image","children": {"qg_image": {"label": "","children": {"image": {"label": "Image","value": [{"in_list": true,"additional_translations": {},"content_type": "image/jpeg","preview_image": "/upload/7f/0/7f06837f-0d6e-428a-999f-

→˓98d3a2f7a115.jpg","key": "Image","value": "/upload/ef/e/efe3d278-6620-49ed-b491-0ce29db214bd.

→˓jpg","template": "raw"

}]

},"image_caption": {"label": "Caption, explanation of photo","value": [{"additional_translations": {},"key": "Caption, explanation of photo","value": "On-farm demonstration and exchange visits: women

→˓demonstrating the system to visitors","template": "raw"

}]

}}

}}

},// ...

}





}}

},// ...

}


Example request with Postman

Obtain Application Token

Return the user’s application token. This application token is used for authorizing the following endpoints:

GET

Fetch Questionnaire Data and List of User’s Questionnaires.

POST

70 Chapter 6. API


Create a New Questionnaire, Upload an Image and, Edit Existing Questionnaire.

Endpoint: /api/v2/auth/login/

Allowed method: POST

Request headers: Content-Type: Specify the format of the POST data. Available formats are JSON(application/json) and XML (application/xml). Default: application/json

Data: email: The QCAT login email of the user. password: The QCAT login password of the user.


Request:

curl -X POST -H 'Content-Type: application/json' -d '{"email":"[email protected]",→˓"password":"abc123!"}' https://qcat.wocat.net/en/api/v2/auth/login/

Response:

{"token":"9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931"

}


Create a New Questionnaire

Return the identifier of the successfully created Questionnaire.

Endpoint: /api/v2/questionnaires/{configuration}/{edition}/create/


Request headers: Authorization: Your application token for the QCAT API. Specified in the format TokenAPP_TOKEN


Content-Type: Specify the format of the POST data. Available formats are JSON (application/json)and XML (application/xml). Default: application/json

Data: data: The questionnaire data.


Request:

curl -X POST -H 'Authorization: Token 9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931' -H→˓'Accept: application/json' -H 'Content-Type: application/json' -d '{"qg_name": [{→˓"name": {"en": "Example_CreateTechnology"}}]}' https://qcat.wocat.net/en/api/v2/→˓questionnaires/technologies/2018/create/

Response:



{"success":"true","code": "technologies_4534"

}


Upload an Image

Return the url and thumbnail information for the uploaded image.

Endpoint: /api/v2/image/upload/




Content-Type: The format of the POST data must be (multipart/form-data)

Data: file: The image file to be uploaded.


Request:

curl -X POST -H 'Authorization: Token 9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931' -H→˓'Accept: application/json' -H 'Content-Type: multipart/form-data' https://qcat.→˓wocat.net/en/api/v2/image/upload/ -F file=@/home/Downloads/example.jpg

Response:

{"success": true,"uid": "cfb23a06-385a-47c5-8a94-83cae1fd90b7","interchange": [

"[/upload/9d/a/9da8b521-7130-48df-ba31-549016a748e5.jpg, (default)]","[/upload/0a/3/0a3fea13-1485-4ec8-92ee-351eef561d2d.jpg, (small)]","[/upload/17/0/170251f9-a9ea-4945-a714-0beaebb7c750.jpg, (medium)]","[/upload/cf/b/cfb23a06-385a-47c5-8a94-83cae1fd90b7.jpg, (large)]"

],"url": "/upload/cf/b/cfb23a06-385a-47c5-8a94-83cae1fd90b7.jpg"}


72 Chapter 6. API


Fetch Questionnaire Data

Returns a questionnaire’s data.

Endpoint: /api/v2/questionnaires/{configuration}/{edition}/{identifier}/

Allowed method: GET




Request:

curl -X POST -H 'Authorization: Token 9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931' -H→˓'Accept: application/json' https://qcat.wocat.net/en/api/v2/questionnaires/→˓technologies/2018/technologies_4534/

Response:

{"status":"draft","data": {

"qg_name": [{

"name": {"en": "Example_EditTechnology"

}}

]// ...

}}


Edit Existing Questionnaire

Return the identifier of the successfully edited Questionnaire.

Endpoint: /api/v2/questionnaires/{configuration}/{edition}/{identifier}/




Content-Type: Specify the format of the POST data. Available formats are JSON (application/json)and XML (application/xml). Default: application/json



Data: data: The questionnaire data.


Request:

curl -X POST -H 'Authorization: Token 9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931' -H→˓'Accept: application/json' -H 'Content-Type: application/json' -d '{"qg_name": [{→˓"name": {"en": "Example_EditedTechnology"}}]}' https://qcat.wocat.net/en/api/v2/→˓questionnaires/technologies/2018/technologies_4534/

Response:

{"success":"true","code": "technologies_4534"

}


List of User’s Questionnaires

Return a list of public/draft Questionnaires where the User is the compiler

Endpoint: /api/v2/questionnaires/mydata/

Allowed method: GET




Request:

curl -X POST -H 'Authorization: Token 9c99bb2d2733f7bea2d20b04ac54b5e0f9fc1931' -H→˓'Accept: application/json' -H 'Content-Type: application/json' https://qcat.wocat.→˓net/en/api/v2/questionnaires/mydata/

Response:

{"results": [{

"name": "Participatory action research on drip irrigation","image": "/upload/22/a/22ad49a2-8e96-45d2-b8a4-bc6ca99f63a3.jpg","definition": {

"en": "Conducting participatory action research with farmers and line→˓agencies for demonstrating, disseminating and scaling up drip irrigation."


74 Chapter 6. API



},"created": "2020-05-05T18:32:51.706944Z","updated": "2020-07-15T08:16:47.710998Z","code": "approaches_2350","edition": "2015","status": "Draft"

},{

"name": "Farmer field schools on integrated plant nutrient systems","image": "/upload/a5/1/a516e41f-f788-4bfd-aacf-fd25899614e5.jpg",

"definition": {"en": "Participatory and collaborative learning through the farmer field

→˓school approach"},"created": "2020-05-05T18:32:51.706944Z","updated": "2020-07-15T08:16:47.710998Z","code": "approaches_2351","edition": "2015","status": "Draft"

},// ...

}


6.1.2 QCAT API v1

Version 1 is not the current version of the QCAT API anymore. Currently, there is no documentation of version 1available.

Please use the current version of the API and refer to its documentation.



76 Chapter 6. API

CHAPTER 7

Hosting

7.1 Hosting

This chapter describes the setup on the servers that host this project. This project can be run on any kind of setup thatsupports wsgi, the following information describes one possible implementation. All configuration files are stored inthe folder serverconfig.

7.1.1 Architecture

Apache is used as reverse proxy; uwsgi serves the application itself.

Apache

The configuration from serverconfig/apache_<environment>.conf may directly be used as symlink inyour allowed hosts. This configuration is built on Apache version 2.4.7 and ensures that:

• All traffic is redirected to use HTTPS

• The communication with the application server is handled with a socket file

Following Apache modules must be enabled:

• a2enmod proxy

• a2enmod rewrite

• a2enmod deflate

• a2enmod headers

The Apache server may be replaced with nginx, if traffic increases heavily.

77

https://httpd.apache.org/

https://httpd.apache.org/docs/2.4/mod/mod_proxy.html

http://nginx.org/en/


Uwsgi

This is the actual application server (see uwsgi docs). It should be run in the emperor/tyrant mode. This configurationis built for uwsgi version 2.0.12.

A sample configuration for this application:

• Emperor: adapt the script: serverconfig/uwsgi_emperor.conf and create a new file in /etc/init.

• Vassals: Symlink (serverconfig/uwsgi_<environment>.conf) in the folder that is ruled by youremperor.

7.2 Latest changes

7.3 Code

7.3.1 Source

The code of QCAT can be found on Github.

7.3.2 Modules

accounts

The Django app accounts handles the authentication and authorization of users.

accounts.authentication

accounts.forms

accounts.models

accounts.urls

accounts.views

configuration

The Django app configuration handles the configuration of the Questionnaires.

configuration.configuration

configuration.models

configuration.views

78 Chapter 7. Hosting

http://uwsgi-docs.readthedocs.org/

https://github.com/CDE-UNIBE/qcat


Notifications

The Django app notifications handles the creation and display of logs for the review process. These are themost important decisions

Signals

• Signals are used to fire events that mark a change in the workflow

• Classic functions could be used, the decision for signals was made because

– More receivers are likely to be implemented soon (e.g. auto-assign reviewers)

– Tasks could be executed asynchronously easily

• All sending signals live in the questionnaire module, notifications receivers in the notifications module.

View

• Heavily built for asynchronous calls. This is due to heavy queries.

• JS is mostly in the file static/notifications/js/notificationActions.js

• Some logic for the filters depends on the get querystring.

Logic

• Most of the logic is in the ActionContextQuerySet, so it is reusable.

• As content of the mails sent to users needs to be more context-specific (e.g. different text when reviewing orpublishing or different text when inviting editor or reviewer), a split between notifications (as in the notificationlist) and mails (as sent to users after certain actions) was made.

Notifications

(as in the notification list)

• The text for a notifications subject (the text appearing in the notification list) depends on its action and is basedon the template templates/notifications/notification/<action>.html

Mails

• No mails are sent yet, as the texts and triggers are not stable.

qcat

The Django app qcat contains the basic settings and functions of the entire QCAT application.

7.3. Code 79


qcat.utils

questionnaire

The Django app questionnaire is responsible for the questionnaires and their versions.

questionnaire.models

questionnaire.upload

questionnaire.urls

questionnaire.utils

questionnaire.views

Summary

This describes the ideas and workflow of how questionnaires are exported as pdf. As of now, there is only one type ofsummary which displays the questionnaire in full - so its not actually a summary but just a pdf-export.

Following changes are planned:

• Users should be able to select/deselect sections in the pdf (description, location, etc.). So editing the markupbefore creating the PDF will be required.

• Changes in the output format (paper formats, fields on display) should be configurable.

Idea and rationale

The summary is not a fixed (hard coded) view, but its content may change depending on configuration and version ofthe config.

The relevant sections for the summary of each questionnaire are defined in the config. Gathering data for the summaryhappens in two steps:

• Parser: get combined data from config and questionnaire (use the same functionality as the questionnaire APIresource)

• Renderer: filter, aggregate and annotate data specifically for the summary

The naming (parser and renderer) is not 100% precise, but indicates the idea.

The created HTML is then converted to PDF with wkhtmltopdf.

This concept was decided upon because:

• Frontend work can be sourced to external colleagues

• Robust handling of data (questionnaires and configs will change)

• More summary types can be added later

• HTML can be converted to different formats (doc) as well

• The very first step of combining data from config and questionnaire was required for another project as well(API)



Technical workflow

• The view `summary.views.QuestionnaireSummaryPDFCreateView` is called

• Data for the summary is created with the defined renderer in `summary.renderers`. This renderer set upthe data by the according to summary type and configuration.

• In the parser module (`summary.parsers`), questionnaire and configuration data is com-bined with the same class built for the questionnaire detail API resource: `configuration.configured_questionnaire.ConfiguredQuestionnaire`

• The full data is then prepared as defined in the configuration.

• The initial idea was to define each question which must appear in the summary. This ‘whitelisting’ is not alwaysa good fit due to repeating questions/ questiongroups. Therefore, specific data preparation methods are availableon the parsers classes.

Add a new summary type

• Either subclass `summary.views.SummaryPDFCreateView` with a custom summary-type, or refactorthe class for dynamic usage via get param or such.

• Define a renderer in `summary.views.SummaryPDFCreateView.render_classes`

• A new template may be created (`summary.templates.layout`) and can be passed as GET-parameterto the view.

• Extend or create the renderer and according templates

Add a new field

• In the fixtures-json file, find the desired question and add the summay-config to the ‘configuration’ value (seeexamples below).

• Depending on the complexity of the question(s) involved, a specific data loading method on the parser may berequired (but is optional).

• For most questions, the default data loading is fine - simply add the field on the respective renderer.

• Minimal example

"configuration": {"type": "radio","summary": { # summary config starts here

"types": ["full"], # list with all summary-types"default": {"field_name": "some_name" # the key which will be used for this data in

→˓the summary}

}}

• Full example

"configuration": {"type": "radio","summary": {


7.3. Code 81



"types": ["full"],"default": {

"field_name": {"qg_31.question_keyword": "field_name_one", # when the same question is

→˓used in multiple questiongroups:"qg_97.question_keyword": "field_name_two" # provide different access-

→˓keys in the summary},"get_value": { # use a custom method on the

→˓parser"name": "get_qg_values_with_scale", # name of the method"kwargs": {"qg_style": "radio" # additional kwargs passed

→˓to the method}

}}

}}

History

• It was planned to pass JSON to the frontend and do all conversion to html with handlebars. This was rejectedfrom we are cube with regard to technical feasibility.

• wkhtmltopdf was selected based on a demo-HTML / proof of concept.

unccd

The Django app unccd contains the configuration for the UNCCD questionnaire.

unccd.views


CHAPTER 8

Indices and tables

• genindex

• modindex

• search

8.1 To Do’s

This page contains a list of all To Do’s that emerged during the development process. The list may be rather unorderedbecause the To Do’s are specified directly in the code.

Todo: Questionnaire configurations can inherit from another one. This needs further documentation.

(The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/qcat/checkouts/latest/docs/configuration/introduction.rst,line 59.)

Todo: This does not really belong here

(The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/qcat/checkouts/latest/docs/development/dataformat.rst,line 129.)

83


84 Chapter 8. Indices and tables

Index

UUser Group, 3User Groups, 3

WWOCAT, 3

85

qcat documentationqcat documentation, release 0.1.0 frontend installation 1.change to project folder...

Documents