AMITaskServer

The ATLAS Metadata Interface Task Server (AMITaskServer) is a generic high level time-based job scheduler. It was originally developed for the A Toroidal LHC ApparatuS (ATLAS) experiment, one of the two general-purpose detectors at the Large Hadron Collider (LHC).

Compiling AMITaskServer

• Requirements

Make sure that Java 11, Maven 3 and AMIMQTTServerJava are installed:

java -version
mvn -version

• Compiling sources

mvn package

Generated standalone bundle: target/AMITaskServer-X.X.X-bundle.zip

Task SQL table

AMITaskServer need a SQL table in order to store job definitions. Supported SGBD: MySQL, PostgreSQL, Oracle.

Schema for MySQL 5 and later:

CREATE TABLE `router_task` (
  `id` INT(11) NOT NULL AUTO_INCREMENT,
  `name` VARCHAR(128) NOT NULL,
  `description` VARCHAR(512) DEFAULT NULL,
  `command` VARCHAR(1024) NOT NULL,
  `commaSeparatedLocks` VARCHAR(512) DEFAULT NULL,
  `locked` INT(1) DEFAULT '0' NOT NULL,
  `oneShot` INT(1) DEFAULT '0' NOT NULL,
  `priority` INT(3) DEFAULT '0' NOT NULL,
  `timeStep` INT(11) DEFAULT '60' NOT NULL,
  `serverName` VARCHAR(128) NOT NULL,
  `running` INT(1) DEFAULT '0' NOT NULL,
  `success` INT(1) DEFAULT '0' NOT NULL,
  `report` TEXT,
  `stdout` TEXT,
  `stderr` TEXT,
  `lastStartDate` INT(11) DEFAULT '0' NOT NULL,
  `lastStopDate` INT(11) DEFAULT '0' NOT NULL,
  PRIMARY KEY (`id`)
);

CREATE TABLE `router_task_server` (
  `id` INT(11) NOT NULL AUTO_INCREMENT,
  `name` VARCHAR(128) NOT NULL,
  `description` VARCHAR(512) DEFAULT NULL,
  PRIMARY KEY (`id`)
);

Schema for SQLite 3 and later:

CREATE TABLE "router_task" (
  "id" INT(11) PRIMARY KEY,
  "name" VARCHAR(128) NOT NULL,
  "description" VARCHAR(512) DEFAULT NULL,
  "command" VARCHAR(1024) NOT NULL,
  "commaSeparatedLocks" VARCHAR(512) DEFAULT NULL,
  "locked" TINYINT(1) DEFAULT '0' NOT NULL,
  "oneShot" TINYINT(1) DEFAULT '0' NOT NULL,
  "priority" TINYINT(3) DEFAULT '0' NOT NULL,
  "timeStep" INT(11) DEFAULT '60' NOT NULL,
  "serverName" VARCHAR(128) NOT NULL,
  "running" TINYINT(1) DEFAULT '0' NOT NULL,
  "success" TINYINT(1) DEFAULT '0' NOT NULL,
  "report" TEXT,
  "stdout" TEXT,
  "stderr" TEXT,
  "lastStartDate" INT(11) DEFAULT '0' NOT NULL,
  "lastStopDate" INT(11) DEFAULT '0' NOT NULL
);

CREATE TABLE "router_task_server" (
  "id" INT(11) PRIMARY KEY,
  "name" VARCHAR(128) NOT NULL,
  "description" VARCHAR(512) DEFAULT NULL
);

Schema for PostgreSQL 10 and later:

CREATE TABLE "router_task" (
  "id" SERIAL NOT NULL,
  "name" VARCHAR(128) NOT NULL,
  "description" VARCHAR(512) DEFAULT NULL,
  "command" VARCHAR(1024) NOT NULL,
  "commaSeparatedLocks" VARCHAR(512) DEFAULT NULL,
  "locked" INT(1) DEFAULT '0' NOT NULL,
  "oneShot" INT(1) DEFAULT '0' NOT NULL,
  "priority" INT(3) DEFAULT '0' NOT NULL,
  "timeStep" INT(11) DEFAULT '60' NOT NULL,
  "serverName" VARCHAR(128) NOT NULL,
  "running" INT(1) DEFAULT '0' NOT NULL,
  "success" INT(1) DEFAULT '0' NOT NULL,
  "report" CLOB,
  "stdout" CLOB,
  "stderr" CLOB,
  "lastStartDate" INT(11) DEFAULT '0' NOT NULL,
  "lastStopDate" INT(11) DEFAULT '0' NOT NULL
);

CREATE TABLE "router_task_server" (
  "id" SERIAL NOT NULL,
  "name" VARCHAR(128) NOT NULL,
  "description" VARCHAR(512) DEFAULT NULL
);

Schema for Oracle 11c and later:

CREATE TABLE "router_task" (
  "id" NUMBER(11) NOT NULL,
  "name" VARCHAR2(128) NOT NULL,
  "description" VARCHAR2(512) DEFAULT NULL,
  "command" VARCHAR2(1024) NOT NULL,
  "commaSeparatedLocks" VARCHAR2(512) DEFAULT NULL,
  "locked" NUMBER(1) DEFAULT '0' NOT NULL,
  "oneShot" NUMBER(1) DEFAULT '0' NOT NULL,
  "priority" NUMBER(3) DEFAULT '0' NOT NULL,
  "timeStep" NUMBER(11) DEFAULT '60' NOT NULL,
  "serverName" VARCHAR2(128) NOT NULL,
  "running" NUMBER(1) DEFAULT '0' NOT NULL,
  "success" NUMBER(1) DEFAULT '0' NOT NULL,
  "report" CLOB,
  "stdout" CLOB,
  "stderr" CLOB,
  "lastStartDate" NUMBER(11) DEFAULT '0' NOT NULL,
  "lastStopDate" NUMBER(11) DEFAULT '0' NOT NULL
);

ALTER TABLE "router_task"
  ADD CONSTRAINT "pk1_router_task" PRIMARY KEY ("id")
;

ALTER TABLE "router_task"
  ADD CONSTRAINT "uk1_router_task" UNIQUE ("name")
;

CREATE SEQUENCE "seq_router_task"
  START WITH 1 INCREMENT BY 1 CACHE 10
;

CREATE TRIGGER "trig1_router_task"
  BEFORE INSERT ON "router_task"
  FOR EACH ROW
  BEGIN
    SELECT "seq_router_task".NEXTVAL INTO :NEW."id" FROM dual;
  END
;

CREATE TABLE "router_task_server" (
  "id" NUMBER(11) NOT NULL,
  "name" VARCHAR2(128) NOT NULL,
  "description" VARCHAR2(512) DEFAULT NULL
);

ALTER TABLE "router_task_server"
  ADD CONSTRAINT "pk1_router_task_server" PRIMARY KEY ("id")
;

ALTER TABLE "router_task_server"
  ADD CONSTRAINT "uk1_router_task_server" UNIQUE ("name")
;

CREATE SEQUENCE "seq_router_task_server"
  START WITH 1 INCREMENT BY 1 CACHE 10
;

CREATE TRIGGER "trig1_router_task_server"
  BEFORE INSERT ON "router_task_server"
  FOR EACH ROW
  BEGIN
    SELECT "seq_router_task_server".NEXTVAL INTO :NEW."id" FROM dual;
  END
;

CREATE OR REPLACE FUNCTION FROM_UNIXTIME(UnixTime IN NUMBER) RETURN TIMESTAMP IS
BEGIN
  RETURN (TIMESTAMP '1970-01-01 00:00:00 UTC' + UnixTime * INTERVAL '1' SECOND) AT LOCAL;
END FROM_UNIXTIME;

Configuring AMITaskServer

Configuration file template (~/.ami/AMI.xml, /etc/ami/AMI.xml or java -Dami.conffile=path/AMI.xml, ...):

<?xml version="1.0" encoding="ISO-8859-1"?>

<properties>
	<property name="server_name"><![CDATA[name]]></property>

	<property name="mqtt_url"><![CDATA[url]]></property>
	<property name="mqtt_username"><![CDATA[username]]></property>
	<property name="mqtt_password"><![CDATA[password]]></property>

	<property name="jdbc_url"><![CDATA[url]]></property>
	<property name="jdbc_username"><![CDATA[username]]></property>
	<property name="jdbc_password"><![CDATA[password]]></property>

	<!--
	<property name="exclusion_server_url"><![CDATA[url]]></property>
	-->

	<!--
	<property name="ping_period"><![CDATA[30]]></property>
	-->
	<!--
	<property name="cleanup_period"><![CDATA[172800]]></property>
	-->
	<!--
	<property name="quality_of_service"><![CDATA[0]]></property>
	-->
	<!--
	<property name="max_nb_of_running_tasks"><![CDATA[10]]></property>
	-->
	<!--
	<property name="priority_probability_ratio"><![CDATA[2.0]]></property>
	-->
</properties>

Using AMITaskServer

vim startAMITaskServer.sh

./AMITaskServer start
./AMITaskServer stop
./AMITaskServer lock
./AMITaskServer unlock
./AMITaskServer status

./AMITaskServer --help

Installing as a Linux service

cp AMITaskServer.service /etc/systemd/system

service AMITaskServer start
service AMITaskServer stop
service AMITaskServer lock
service AMITaskServer unlock
service AMITaskServer status

Deploying with Docker

See official image there.

Task environment variables

The following environment variables are available in the task processes:

• AMI_TASK_ID: the task id
• AMI_TASK_MAME: the task name
• AMI_TASK_COMMAND: the command line
• AMI_TASK_REPORT_FILE: path where to write the task report

Developers

• Jérôme ODIER (CNRS/LPSC)
• Fabian LAMBERT (CNRS/LPSC)