wiki:Service_Lab_Plugin

Service Lab Documentation

Service Lab has a GRITS installation in its lab. This installation has some additional features added specifically for the lab. It is a short documentation of additional functionalities provided to the lab and their management in GRITS.

Plugins

GRITS provides some specialized functionalities for Service Lab. Currently there are two plugins that have been created for this purpose.

Service Lab main plugin

This plugin provides functionalities related to accounting and management in Service Lab. This plugin should be added to the product file when creating a deployment for the lab. Currently this plugin has following functionalities:

  • Invoice Generation
    • price management for protocols and generation of invoice for projects
  • Project Validation
    • Task assignment to individual samples in projects, verification of experiments in samples and files in archives
  • Management of Task and File Upload information
  • Upload of final Bill for projects

This plugin also contains

  • the set of latest experiment designer's ontology provided to Service Lab
  • the protocol variants provided to Service Lab
  • product files that generate GRITS products for deployment
    • GRITS.servicelab.product - to deploy in lab
    • GRITS.servicelab.dev.product - for developers use before deployment (has the additional developers fragment)

Service Lab utility plugin

This plugin is used for generation of GRITS workspace projects from spread sheets. It is also used for adding new protocols to ontology from a spread sheet and for generating its protocol variants. This plugin should not be added to any product file that would be deployed for use. Currently this plugin adds two commands to GRITS in the popup menu of the project explorer:

  • Import Projects
  • Add Ontology

This plugin contains

  • Protocol spread sheet for generation of ontology
    • name would be Protocol Set <SET_NUMBER> Rev<REVISION_NUMBER>.xlsx
  • Sample spread sheet for creating descriptors from sample names while generating workspace
  • a product file GRITS.all.lab.service.product for ontology and workspace generation.

Generated ontology

  • Service Lab uses an ontology that is an extended version of the ontology in the experiment design plugin. To keep track of the version of ontology that was used to build the ontologies for Service Lab the version number is given an increment of 100 to the ontology of experiment design. For example - Service Lab ontology version 101.24.0 would mean it used 1.24.0 ontology from experiment design plugin to build the ontology for Service lab.

Note: Service Lab has changed format of its project spread sheet (spread sheets from which workspace is generated) several times. These were indicated by a number type in source code packages. The most recent type that was used for generating workspace was type 3.

Note: For data privacy, the spread sheet containing project information for generating workspace is not added to the svn and is used locally. For generating workspace from the 3rd type (latest version) spread sheet, the spread sheet should be placed in the doc folder of this plugin.

Tutorials

GRITS Users

Useful tutorials for Service Lab users can be found here:

Developers

Service Lab Main Plugin

It is broadly divided into 4 parts based on current functionalities.

  • Invoice Generation
  • Project Validation
  • Management for Invoice Generation and Project Validation information
  • Bill Upload

Also important thing to note is that there are ${SERVICELAB_FOLDER} at two levels that are used for storing Service Lab information in a workspace. ${SERVICELAB_FOLDER} has its name stored in IConfig.SERVICE_LAB_FOLDER_NAME.

Location :

  • Service Lab folder in workspace - ${WORKSPACE_FOLDER}/ ${SERVICELAB_FOLDER}
  • Service Lab folder for a project - ${WORKSPACE_FOLDER}/ ${PROJECT_FOLDER}/ ${SERVICELAB_FOLDER}

Note:2nd level of Service Lab folder inside each project is used to independently store specific Service Lab information of projects and to avoid update issues when projects are renamed or removed.

Invoice Generation

When a invoice is to be generated, the handler class gets invoked. This class looks for Protocol Price file in the .servicelab folder and passes the required information to a Invoice wizard for further processing. Once the information has been edited/confirmed by user for download, it saves the invoice as MS Word file to a location that is returned by the wizard. Important information to note :

  • Protocol Price file
    • File Name - IConfig.PROTOCOL_PRICE_INFO_FILE_NAME
    • File Location - ${WORKSPACE_FOLDER}/ ${SERVICELAB_FOLDER}/ ${PROTOCOL_PRICE_FILE}
  • Handler - handler/GenerateBillHandler
  • Wizard - wizard/invoice/InvoiceWizard
  • Price calculation - invoice/ProtocolPriceCalculator
  • Saving as MS Word - invoice/MSWordInvoiceGenerator
Project Validation



Steps for validating a project

To validate a project the handler class gets invoked. This class looks for Task Protocol file and File Upload Information file in the .servicelab folder and passes the required information to a wizard for further processing. Important information are summarized below :

  • Task Protocol file
    • stores protocol information for tasks
    • File name - IConfig.TASK_PROTOCOL_INFO_FILE_NAME
    • File location - ${WORKSPACE_FOLDER}/ ${SERVICELAB_FOLDER}/ ${TASK_PROTOCOL_FILE}
  • File Upload Information file
    • stores file upload information for protocols
    • File name - IConfig.PROTOCOL_FILE_UPLOAD_INFO_FILE_NAME
    • File location - ${WORKSPACE_FOLDER}/ ${SERVICELAB_FOLDER}/ ${FILE_UPLOAD_INFO_FILE}
  • Handler - handler/VerifyExperimentsHandler
  • Wizard - wizard/validation/ValidationWizard

Various classes for matching and validation algorithm (inside "validation/" package)

  • ProjectTaskAssigner
    • for assigning tasks to samples
    • assigns tasks to each sample equally (if number of tasks is not divisible by number of samples then leaves all the tasks for user to assign)
    • previous assignment by user is also stored in a file and is given priority for re-assignment
      • class - PreviousTaskAssignmentsHandler
      • Previous Assignment File
        • name - IConfig.PROJECT_TASK_ASSIGNMENT_FILE_NAME
        • location - ${PROJECT_FOLDER}/${SERVICELAB_FOLDER}/${PREVIOUS_ASSIGNMENT_FILE}
  • SampleProtocolTaskMatcher
    • for matching task to protocols of each sample
    • matches protocols to its possible tasks (uses backtracking to try all possible combination unless each task has valid number of protocols and there is no extra protocol)
  • ProtocolFileMatcher
    • for matching files in project archive to protocols in samples
    • matching is done for uploaded file Category and SubCategory
    • makes sure all protocols have their required number of files available in Project archive
    • no errors if extra files are there in the archive
Management for Invoice Generation and Project Validation

The three files (Protocol Price file, Task Protocol file and File Upload Information file) described in the above two sections are used for managing invoice and project validation information and are located in the ${SERVICELAB_FOLDER} folder of the workspace. These files are managed by three Managing Editor Parts

  • PriceManager (part/pricemanager/ProtocolPriceManager)
  • TaskProtocolManager (part/taskmanager/TaskProtocolManager)
  • FileUploadManager (part/filemanager/ProtocolUploadFileManager)

Each of the three managing editors have only one instance (not Multiple option selected in the fragment.e4xmi file) in the application. These managers reload their information from workspace each time they are opened by a user. To learn more about them also read Project Validation Tutorial.

In the current implementation some of the important functionalities that are missing are:

  • Delete manager files
  • Import protocol/task information from files of other workspace
Bill Upload

The final bill file can be uploaded for each project. The bill file is stored in the archive of the project. The bill amount is also stored in a Project Statistics File for each project.

  • Project Statistics File
    • contains a map of statistic variables and their values
    • name - IConfig.PROJECT_STATS_FILE_NAME
    • Location - ${PROJECT_FOLDER}/${SERVICELAB_FOLDER}/${PROJECT_STATS_FILE}
  • Handler - handler/UploadBillHandler
  • Dialog - dialog/UploadBillDialog

Service Lab utility plugin

Code in Service Lab utility plugin should be updated from time to time for compatiblity with other plugins as it uses different plugins classes and functionalities for performing its tasks. Currently it does two tasks

  • Ontology generation
  • Workspace generation
Ontology Generation

The handler class passes the spread sheet that contains the protocols templates to be added in the ontology. It looks in the doc folder of its plugin to find this spread sheet. The main class for importing ontologies first copies the standard experiment design ontology (experimentdesignontology.owl) from experiment design plugin as a local ontology (localexperimentdesignontology.owl) in the config folder (.grits-toolbox/org.grits.toolbox.editor.experimentdesigner/). It adds protocol templates to this ontology and generates protocol variants in the config folder. It then copies back the local experiment design ontology as standard ontology in the experiment design plugin.

Sometimes excessive HTTP requests for reading Pubmed papers may result in server blocking the operation. To overcome this, one can download the papers using browser and save it in the paper folder (name being PubmedId of the paper and file type being ".xml"). Codes have been adjusted to read the papers from this paper folder if response from HTTP request has certain error phrases.

  • Handler - handler/AddOntologyHandler
  • package to look into - addtoontology/
  • main class - addtoontology/ImportTemplates (short name for import protocol templates)

Important locations

  • .grits-toolbox/org.grits.toolbox.editor.experimentdesigner/ is the config folder (for writing to local ontology and generating protocol variants)
  • ontology folder in experiment design plugin (copies standard ontology as local ontology and once generation is complete, copies back to this folder as standard ontology)
  • namespace folder in experiment design plugin (adds short namespaces for parameters that were created while adding protocols in the ontology)
  • doc folder in its own plugin (for protocol spread sheet)
  • paper folder in its own plugin (for offline reading of Pubmed papers)

After ontology is generated, for archiving the generated ontology and protocol variants one has to manually

  • Place the experimentdesignontology.owl from experiment design plugin into the Service Lab Main plugin. Also update its version by adding 100 to the version of the original ontology in experiment design that was used for generating this ontology
  • Also add a copy of the local ontology localexperimentdesignontology.owl with the same version as experimentdesignontology.owl for compatibility
  • Compress the protocol variants in config folder and set version as that of the ontology that is generated to show compatibility and place this in the protocol variants folder of the Service Lab Main plugin
  • update these files in svn
Workspace Generation

For generating a workspace to be used with GRITS, this plugin can also read information from a spread sheet and create a workspace that has

  • project
  • sample
  • experiment design

Important classes and package

  • Handler - handler/ImportProjectsHandler
  • package to look into - importprojects/sheet_type3/ (for latest sheet)
  • main class - addtoontology/sheet_type3/ImportProjectsSheetType3 (for latest sheet)

One can also roll back to previous sheet type 2 by changing the ImportProjectsHandler to call org.grits.toolbox.lab.service.importprojects.sheet_type2.ImportProjectsSheetType2 (this package has no compile error and should work but has not been tested recently for generating any workspace) instead of calling org.grits.toolbox.lab.service.importprojects.sheet_type3.ImportProjectsSheetType3

For generating the workspace from latest spread sheet:

  • Copy the spread sheet (containing project information) in the doc folder of the Service Lab utility plugin
  • update the value for variable PROJECT_SPREAD_SHEET in ImportProjectsSheetType3 class
  • Run the GRITS.all.lab.service.product product from Service Lab utility plugin
  • Select or create a new workspace while starting GRITS application
  • Right click on popup menu in the project explorer
  • Select Import Projects
  • Wait for projects and samples to be created and then close GRITS

In the first run it will create a workspace with only projects and sample. Once the values are verified, one should make a backup of this workspace so that one does not have to start from scratch if next steps are not done properly. Then one can add experiments to this workspace by following the steps:

  • delete localexperimentdesignontology.owl from experiment design config folder
    • else could result in duplicate error as it recreates protocols for Temporary Category
  • Re-run the GRITS.all.lab.service.product product from Service Lab utility plugin
  • While starting GRITS select the previous workspace that has projects and samples
  • Right click on popup menu in the project explorer
  • Select Import Projects
  • Wait for experiments to be added and then close GRITS

This would add the experiments in the existing workspace with projects and samples.

One can also change the combinations of generating workspace to create project and add sample and experiments later (and other such combinations) by changing the enum arguments in ImportProjectsHandler.

If some information were not correct and workspace needs to be regenerated then first clean the workspace and then redo the above steps. For cleaning and regenerating the workspace :

  • Set the variable DEFAULT_WORKSPACE in ImportProjectsHandler class to this workspace that is to be cleaned
  • Run the ImportProjectsSheetType3 class as Java application (when its main method does not receive any argument then it cleans the DEFAULT_WORKSPACE)
  • then repeat the steps that were described above for creating projects with samples and experiment designs

Deployment

Following steps are to be followed each time GRITS is to be installed in Service Lab:

  1. Copy two ontologies before exporting
    • copy the two ontologies from ontology folder of Service Lab main plugin to ontology folder of Experiment designer plugin
      1. experimentdesignontology.owl
      2. localexperimentdesignontology.owl
  2. Make sure the namespace file for Service Lab protocols (namespaces_servicelab.xml) is there in the experiment design plugin inside namespace folder
  3. Use the product "GRITS.servicelab.product" in Service Lab Main plugin to export a product as one would normally export
  4. Test and deploy the exported product in Service Lab computer

Note: Service Lab has its own set of ontologies and protocol variants. It cannot work merely with the experiment design ontologies. Each time a product is made for Service lab, the ontologies from the Service Lab Main plugin should be copied to build the product.

Currently one can only generate a new set of protocol variants (it would remove all local protocols and protocol variants) and one do not have the option for adding protocol variants to the existing protocol variants. If a new set of protocol variants have been generated for Service Lab and need to be installed then follow the following steps :

  1. Get a copy of protocol variants from protocol variants folder of Service Lab main plugin (use the protocol variants with matching version of the ontology that was used in step 1 for deployment)
  2. In Service Lab computer, clean the org.grits.toolbox.editor.experimentdesigner folder inside config folder and copy the new protocol variants and extract it here
Last modified 5 years ago Last modified on 03/20/2017 03:09:05 PM

Attachments (1)

Download all attachments as: .zip