Referencing a Common File in Multiple Projects

This document provides instructions for referencing a common file between projects using the Decipher hooks system.

 Requires Decipher Cloud

Note:  The examples in this article make use of the survey_environment hook. Contact Forsta Support for assistance with custom hooks.

When programming surveys, the need may arise to have access to the same set of participant information in different projects. For example, when reusing participants in multiple studies, or when validating participant responses against a predefined list of responses.

The steps below will achieve this setup:

• Create and upload your data file.

• Create and modify an embedded.py file in your company directory.

• Create and modify a hooks.py file in your company directory.

1: Data File Format

Before you start programming your file referencing template, you need to clarify what file types you can upload and reference with this method. Your file should be a tab-delimited .dat file. To do this, select "Save as" in the raw data sheet, and choose the tab-delimited format. In Microsoft Excel, there is an option for this. When saving your file, you can give it any name you want (in this example, the file is saved as participant_data.txt). Then, edit your file name and manually replace ".txt" with the ".dat" extension and save. The result should be participant_data.dat.

Each data point in your file should be a separate column, and each column should have a header that is a lower-cased single word.

Your file should also have a column with a unique identifier for each participant. This way, the system can pull the unique identifier for your participants when they enter the survey and match it against your data file. It is recommended to use source as your unique participant identifier, as this variable is automatically generated by the system. If you want to use a different variable as a unique identifier, you can add a participant source variable to your survey.

1.1: Uploading the File

To use this file across multiple projects, first upload it somewhere that every project can access it. Using an SFTP client such as FileZilla, you can upload your file to your company-level directory. For example, if your projects selfserve/53b/160800 and selfserve/253b14e/160801 will be using this file, upload it in selfserve/53b.

2: The embedded.py File

The embedded.py file is a Python script file that can be used to embed pre-defined functions within your survey_environment hook. To begin, paste the following code in the first line.

from hermes.syslib.indexed import File as _File

The code above imports the Python File I/O module that handles loading the file into the system and opening it for reading.

Note:  For security reasons, when trying to import different modules within the embedded.py file, always import them with a “_” prefix (e.g., import File as _File , as in the example above).

After importing the module that you need, define the function that loads the file.

def load_file():
 return _File('selfserve/53b/participant_data.dat',key="source",smashCase=True)

The name of the function can be whatever you need, but for this example, it is defined as load_file(). Within this function, you can call the File function with three arguments:

'selfserve/53b/participant_data.dat'	The path to your file.
key="source"	The unique identifier used for the look-up.
smashCase=True	Ensures that the system follows the letter-casing of the key (i.e., it respects that abcDEF is different than ABCdef).

Once you have created your file, it should look like the below:

from hermes.syslib.indexed import File as _File
def load_file():
 return _File('selfserve/53b/participant_data.dat',key="source",smashCase=True)

There are three main things you need to modify here - the name of your function, the file path for your data file, and the unique identifier variable you will use (i.e., your key).

2.1: Modifying the function name

In the template above, the function was defined using def load_file():. To change the name of your function, replace load_file with the name of your preference. For example, if you want to change the name of your function to "loader", replace def load_file(): with def loader():.

Note:  When declaring functions in Python, use standard Python naming conventions - i.e. camel-cased names with no spaces, using only alphanumeric characters.

2.2: Modifying the file path

To modify the file path, replace the first argument in the _File() function on the last line of your template.

return _File('selfserve/53b/participant_data.dat',key="source",smashCase=True) 

Using the example above, replace selfserve/53b/participant_data.dat with selfserve/myCompany/myFile.dat.

2.3: Modifying the key

To modify the key that you will use to pull data, you can change source in key="source" to reference any unique extraVariable or participant source variable you have in your survey (e.g., key="myVariable").

2.4: Modified template

Once you have finished the modification to your code, it should look like the below:

from hermes.syslib.indexed import File as _File
def def loader():
 return _File('selfserve/myCompany/myFile.dat',key="myVariable",smashCase=True)

3: Preparing the hooks.py file

The hooks.py file is a file that can be used to define global variables or functions you want to be executed at different stages of your survey. In this example, you will be using the survey_environment hook, which works similarly to how an <exec when="init"> block would work. The survey_environment function is a Python function within which you input the code to execute your embedded.py file.

def survey_environment(env,survey):
       e={}
       execfile('selfserve/53b53/embedded.py',e)
       env['load_file']=e['load_file']

Important: The survey_environment function takes two arguments, which should not be changed.

env	The survey environment into which you are adding Python functions
survey	The survey object for which data is being generated

Within this function, you call in your embedded.py file and pass on the function you defined in that file to your survey environment.

Note: When passing functions from embedded.py into your survey environment, encapsulate those functions in a separate environment. This will otherwise trigger security measures, and all projects in which this hook applies will error out.

Within line 2 of the survey_environment function, create your own environment to run the embedded.py file. Line 3 will execute this file within your environment. Then, in line 4, pass the load_file function from the custom environment (e) to the survey environment (env), which will make it run automatically within your survey.

3.1: Creating the hooks.py file

To create the hooks.py file, navigate to your company directory and follow the same procedure as with the embedded.py file.

vim hooks.py
Add changes
:wq!

3.2: Modifying the survey_environment function

Below is the template for the hooks.py file, which includes the survey_environment hook.

def survey_environment(env,survey):
       e={}
       execfile('selfserve/53b/embedded.py',e)
       env['load_file']=e['load_file']

In this template, there are only two things that you need to modify - the path to your embedded.py file, and the name of the function that you will load into your hook.

3.2.1: Modifying the embedded.py path

If you want to change the path to your file, change the first argument within your execfile() function. For example, changing 'selfserve/53b/embedded.py' to 'selfserve/abc/embedded.py' would look like the following:

execfile('selfserve/abc/embedded.py',e)

One thing to note here is that the second argument of the function, "e", remains unchanged.

3.2.2: Modifying the function to reference

To change what function within your embedded.py file you are using, you can just change that name on line 4.

env['load_file']=e['load_file']

If your function was called file_loader, then you would change the above line of code to the following:

env['file_loader']=e['file_loader']

If you have multiple functions defined within your embedded.py file, you can add them to your survey environment using the same syntax as above.

env['function1']=e['function1']
env['function2']=e['function2']
env['function3']=e['function3']
env['function4']=e['function4']

3.3: The modified template

After the above modifications, this is what your hooks.py file should contain:

def survey_environment(env,survey):
       e={}
       execfile('selfserve/abc/embedded.py',e)
       env['file_loader']=e['file_loader']

4: Accessing the file from a survey

Once you have set up your embedded.py and hooks.py files, you will be able to use the custom function you defined using an <exec> block within your survey.

<exec when="init">
file = load_file()
</exec>

Using the when="init" attribute will ensure that your load_file() function is only executed once. See Exec Tag to find out more about the when attribute.

Note: You do not need to pass in any arguments to reference the location of your file, as that is already handled when defining the load_file() function inside the embedded.py file.

After loading your file, you can access the data for a specific record within that file using the following code:

<exec>
p.resp_info=file.get(source)
</exec>

In this case, source is being used as the look-up variable, but you can modify that to be any other extraVariable or participant source variable, depending on how you have setup your load_file() function.

Note: You should also assign resp_info to a persistent variable, so that it is accessible throughout your survey.

Once you get the information for the current record, you can access specific data fields from your file and assign them to hidden questions or base logic off of them.

4.1: Storing the Information in a Hidden Question

Using the file example from section 1 above, you can assign a participant's first name in a hidden question by accessing the fname field in your resp_info variable.

<exec when=”init”>
file = load_file()
</exec>
<exec>
p.resp_info=file.get(source)
q1.val = p.resp_info['fname']
</exec>
<text 
 label="q1">
 <title>Participant's First Name</title>
</text>

4.2: Using the Information for Conditions

You can also use the participant record object to create survey logic. As long as you have your record data stored in a variable, you can reference it in logic. Using the same example file from section 1 above, you can base your logic on each participant’s segno variable.

<exec when=”init”>
file = load_file()
</exec>
<exec>
p.resp_info=file.get(source)
</exec>
<html label=”cm1” where=”survey” cond=”p.resp_info[‘segno’] == ‘1’”