SAGrid Application Porting Procedure

Introduction

This area outlines the process to port scientific applications to the South African National Grid ( SAGrid ) infrastructure. In order to be executed on the grid sites, the applications need to be installed on the CE's. All applications should be installed on all CE' and should be exactly the same, so some centralisation of the installation and tagging procedure is necessary. This guide explnis this procedure.

We assume that the applications are available either in a private repository (that of SAGrid), or in a publicly hosted server, sch as the application or project website.

Step 1

Obtain the source application and relevant dependencies required for a successful compile and install. Ensure that all tests are conducted on Scientific Linux platform for consistency. This will ensure that results are as accurate as possible when deployed to other grid sites.

Step 2

Application installs file will be defined in the LFC. Update the LFC with the applications dependencies.The reason we are not making them available online is to circumvent the proxy authentication servers which some sites are connected to. Grid jobs fail due to the application source not being available via HTTP. Another reason for us hosting it on the LFC was to ensure that the versions hosted, worked. Export the following as listed below. This will allow connectivity to the LFC using the LCG commands.

Export the following to your shell

• export LFC_HOST=devslngrd002.uct.ac.za
• export LFC_TYPE=lfc

Create a directory on the LFC to store the contents of your application

• lfc-mkdir /grid/sagrid/software/application_folder
• lfc-ls /grid/sagrid/software/

Register the application install files in the LFC and upload to the SE

Lets find a SE to host our application data

• lcg-infosites --vo sagrid se

• lcg-cr --vo sagrid -d srm://grid-se.ufs.ac.za:8443/data/sagrid/application_folder/application_name.tar.gz -l lfn:/grid/sagrid/software/application_folder/application_name "file://$PWD/application_name.tar.gz"

Once the application has been registered in the logical file catalogue the LFC service will return a GUID. For example: guid:624d679c-c81e-4054-9031-f4d5cd1b8f2b

Confirm the files are registered -

• lfc-ls /grid/sagrid/software/application_folder

Step 3

Preparing the JDL and installation scripts to execute on the remote grid enabled worker nodes. Examples of these can be found JDL and Install_Script. If you plan to use these examples, ensure that references to the application are replaced with your own.

Step 4

VO authentication

Testing your application install is important so we suggest that you execute a test installation job against one of the grid sites.

• Authenticate to the VO as a software manager to deploy the application. If you have not been added to the software manager role then contact the co-ordinator of the South African National Grid. Command: voms-proxy-init --voms sagrid:/sagrid/Role=SoftwareManager
• " voms-proxy-info -all " will display the following below.
  
• ===VO sagrid extension information ===
  VO : sagrid
  subject : /C=IT/O=INFN/OU=Personal Certificate/L=ZA-UCT/CN=Timothy John Carr
  issuer : /C=IT/O=INFN/OU=Host/L=ZA-UFS/CN=voms.virt.sagrid.ac.za
  attribute : /sagrid/Role=SoftwareManager/Capability=NULL

Next, you are required to check your code into the SVN respository located at http://ops.sagrid.ac.za/repo/trunk/demo. This is a area for all test JDL and installation code before migration to the production SVN location. Please adhere to these instructions.

SVN Script Management

First we will need to checkout the project onto our local machine.

• svn checkout https://ops.sagrid.ac.za/repo/trunk/demo

Create a new application folder within /repo/trunk/demo/to keep track of your scripts. Ensure that you are within the /repo/trunk/demo folder on your local machine after the checkout to create the directory else it will error out complaining about .svn entries missing.

• svn add application_folder

Add your JDL / installation scripts to the folder. Execute the following command to commit

• svn add file.jdl

Happy?, if so you may commit your changes.

• svn commit

By default EMACS will open for you to update with a brief commit description. Feel free to change the default editor variable here - EDITOR="emacs -nw" to VI or any editor you feel comfortable with. For those non-EMACS users you can enter the description at the top, press "ctrl+x" and then "ctrl+s" to save, then "ctrl+x" and then "ctrl+c" to exit EMACS. This will commit the change to https://ops.sagrid.ac.za/repo/trunk/demo/application_folder and is viewable at this URL.

Installing the application via JDL

Right, lets deploy the application. Ensure that you have authenticated to the VOMS and particularly the SAGrid VOMS.

The command below will submit the job to the WMS with a auto-generated job ID logged to the file "application-install-JID.log"

• "glite-wms-job-submit -a -o application-install-JID.log application.jd"

• "glite-wms-job-status -i application-install-JID.log"
  

or

• " glite-wms-job-status -v 2 -i application-install-JID.log"

Step 6

Tag the Application at a Grid site

On successful deployment of the application you may then deploy to the rest of South Africa. Currently sagrid deploy applications using a per site requirement attribute available in the example JDL.

Tagging a site requires that you again have the Software Management role extension as part of your initial authentication. The tag below will tag an application named " Auto07p " with the application tag naming convention "VO-sagrid" at a computing element which is registered at ZA-UCT-ICTS.

Command: "lcg-tags --ce srvslngrd004.uct.ac.za --vo sagrid --add --tags VO-sagrid-Auto07"

Retrieving a further list of computing elements can be found using the command: "lcg-infosites --vo sagrid ce"

Conclusion:

This concludes the documentation for application porting on SAGrid Infrastructure.

-- TimothyCarr - 13 Sep 2011