Configuration files description

Description

This page describes configuration files (syntax, format) used by each service in Redcurrant. SysAdmins should take time to understand the different services and their configuration in order to proper plan Redcurrant installation. In the following sample, please replace $NAMESPACE and $HOSTNAME by your own appropriate values.
Redcurrant advised filesystem hierarchy is described in the following table.

Level 1 Level 2 Level 3 Level 4 Role
/GRID Configuration and log file root dir
/GRID /common Common to all nodes. ie: gridagent
/GRID /common /conf Configuration files
/GRID /common /logs Log files of gridagent
/GRID /common /run Unix socket for inter process communication
/GRID /$HOSTNAME Common to all services for this host
/GRID /$HOSTNAME /conf Configuration files
/GRID /$HOSTNAME /conf /gridinit.conf.d Configuration files
/GRID /$namespace Common to all services for this namespace
/GRID /$namespace /$HOSTNAME Common to all services for this namespace
/GRID /$namespace /$HOSTNAME /conf Service configuration
/GRID /$namespace /$HOSTNAME /core Core dump location
/GRID /$namespace /$HOSTNAME /init To be executed before starting services
/GRID /$namespace /$HOSTNAME /logs Services log files
/GRID /$namespace /$HOSTNAME /run Pid files and apache shm files
/DATA Data directory root dir
/DATA /$namespace Data directory for services in this namespace
/DATA /$namespace /$HOSTNAME Data directory for services in this namespace on this hostname
/DATA /$namespace /$HOSTNAME /$service Data directory for this service in this namespace on this hostname
  • Redcurrant packages automatically create user/group (admgrid:admgrid)
  • Redcurrant services rely on log4c library for log configuration. Therefore, each service is configured with one .conf file and one .log4crc file in the same directory
  • Services data are usually located in /DATA root dir as configued in file on through command line options
  • Be sure to give write permission to the process user/group (usually admgrid/admgrid) to data dir and log dir

Namespace

Even if a namespace is not really a service, it must be declared on each Redcurrant nodes (server) and client. Namespace declaration requires conscience and zookeeper addresses. The configuration file is located in /etc/gridstorage.conf.d/.

[MY_NAMESPACE1]
conscience=my_conscience_ip1:my_conscicence_port1
zookeeper=my_zoo_ip1:my_zoo_port1[,my__zoo_ip2:my_zoo_port2]*
  • Reference each namespace in its own file
  • Namespace entry point is the conscience address. Each node must reference the same ip:port.
  • Zookeeper is used by most Redcurrant services. Zookeeper is a distributed service, several instances can be declared with a comma separator. It is advised to run an odd number of Zookeeper to ensure quorum integrity.

Gridinit

Gridinit is a tool used to manage Redcurrant services on a node. Its role is to fork and monitor, like init process on a *NIX OS. Gridinit only manages processes that does not daemonize. Gridinit configuration can be found in /GRID/$HOSTNAME/conf/gridinit.conf.

Usage

gridinit [OPTIONS] ... CONFIG_PATH [LOG4C_PATH]

Options:

-d Detaches then daemonizes the gridinit
-h Displays this help section
-g Limits the services loading to those belonging to the specified group. This option can be repeated.
-q Quiet mode, suppress non-error output

Configuration

Gridinit configuration is done using a .ini-like config file parsed by glib (https://developer.gnome.org/glib/stable/glib-Key-value-file-parser.html).
It contains only one group: [default]

Directives

Name Type Meaning Default value
listen string Location of Unix socket for interprocess communication /GRID/common/run/gridinit.sock
pidfile string Location of the pid file none
groups_only string A list of service groups (see service configuration) to limit gridinit actions none
working_dir string Location of the “Current Working Dir” used by forked processes none
user string The username used to run forked processes none
group string The groupname used to run forked processes none
uid integer The userid used to run forked processes (overrides directive user) none
gid integer The groupid used to run forked processes (overrides directive group) none
limit.stack_size integer The thread stack size to apply to forked processes (in kbytes) 1024
limit.core_size integer The core size limit to apply to forked processes (in blocks) unlimited
limit.max_files integer The maximum number of open fd to apply to forked processes 8589934592
include string Location of service declaration none

Example

gridinit.conf (Replace $HOSTNAME with the server name)

[Default]
listen=/GRID/$HOSTNAME/run/gridinit.sock
pidfile=/GRID/$HOSTNAME/run/gridinit.pid
working_dir=/GRID/$HOSTNAME/run

uid=admgrid
gid=admgrid
limit.core_size=-1
limit.max_files=8192
limit.stack_size=256

include=/GRID/$HOSTNAME/conf/gridinit.conf.d/*

gridinit.log4crc

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE log4c SYSTEM "">
<log4c>
    <config>
        <bufsize>2048</bufsize>
        <debug level="4"/>
        <nocleanup>0</nocleanup>
    </config>

    <category name="root" priority="info" appender="RC,$HOSTNAME,common,gridinit"/>
    <appender name="RC,$HOSTNAME,common,gridinit" type="syslog" layout="basic"/>
    <layout name="basic" type="basic"/>
</log4c>

Service configuration

Each service managed by gridinit can be declared in a dedicated configuration file and stored in /GRID/$HOSTNAME/conf/gridinit.conf.d/ This configuration file is a .ini-like config file parsed by glib (https://developer.gnome.org/glib/stable/glib-Key-value-file-parser.html).
It contains only one group: [Service.<svc_id>] with svc_id usualy $NAMESPACE-$HOSTNAME-$SERVICETYPE-$INSTANCE

Directives

name type possible values role
command string any valid command line Command line to be forked by gridinit, with options
enabled boolean true|false If false, the service is not started and appears as disabled in gridinit_cmd
start_at_boot boolean yes|no If yes, this service will start whenever gridinit start
on_die string respawn|cry|exit If respawn, gridinit will try to restart maximum 4 times the process if it dies unexpectedly
group string comma separated strings Group is a string for group action in gridinit_cmd command
env.PATH string valid PATH envars Restrict PATH env var to this value before forking command
uid integer any valid uid on the server The userid used to run forked processes (overrides value defined in gridinit.conf)
gid integer any valid gid on the server The groupid used to run forked processes (overrides value defined in gridinit.conf)
limit.stack_size integer The thread stack size to apply to forked processes (overrides value defined in gridinit.conf)
limit.core_size integer The core size limit to apply to forked processes (overrides value defined in gridinit.conf)
limit.max_files integer The maximum number of open fd to apply to forked processes (overrides value defined in gridinit.conf)
working_dir string any valid path Location of the “Current Working Dir” used by forked processes (overrides value defined in gridinit.conf)

Example

[Service.$NAMESPACE-$HOSTNAME-meta0-1]
command=/usr/local/bin/meta0_server -v -p /GRID/$NAMESPACE/$HOSTNAME/run/meta0-1.pid -s RC,$NAMESPACE,$HOSTNAME,meta0-1 -O Endpoint=$ip:$port $NAMESPACE /DATA/$NAMESPACE/$HOSTNAME/meta0-1
enabled=true
start_at_boot=yes
on_die=respawn
group=$NAMESPACE,$HOSTNAME,meta0
env.PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

Start Gridinit

Reference gridinit in your OS runlevel. For Redhat/CentOS/Fedora:

chkconfig --add gridinit
chkconfig gridinit on

Gridagent

Gridagent is a kind of proxy which relays services statistics from the node to the conscience.
Gridagent is started by gridinit and has its own configuration file /GRID/comon/conf/gridagent.conf.

Usage

gridagent OPTIONS CONFIG_PATH [LOG4C_PATH]

Options:

–help Displays this help section
–child-req Start only the core agent
–child-evt $NAMESPACE Start only the event manager for the specified namespace
–supervisor Start a supervisor agent which will fork a core agent and an event manager per namespace

Configuration

Gridagent configuration is done using a .ini-like config file parsed by glib (https://developer.gnome.org/glib/stable/glib-Key-value-file-parser.html).
It contains two groups: [General] and [server.inet]

[General] Directives

Name Type Meaning Default value
user string username used to start gridagent none
group string groupname used to start gridagent none
unix_sock.gid integer uid to use for the unix communication socket process uid
unix_sock.uid integer gid to use for the unix communication socket process gid
unix_sock.mode string permissions to use for the unix communication socket 660
backlog integer (128<i<1024) UNIX and TCP backlogs 256
Name Type Meaning Default value
service_check_freq integer local redcurrant services TCP check freq (in sec) 30
enable_broken_elements boolean Allows on
period_broken_push integer Period applied to the periodic upstream of broken elements 2
period_broken_get integer Period applied to the periodic downstream of broken elements. 2
cluster_update_freq integer Default value for the next 5 variables 5
period_get_ns integer Period applied au periodic reloads of all the namespace info. 5
period_get_srv integer Period applied to all periodic reloads of service items. 5
period_get_srvtype integer Period applied to all periodic reloads of services types. 5
period_get_evtconfig integer Period applied to all periodic reloads of routing configuration of meta2 events. 5
period_push_srv integer Period applied to all pushes of local services to the conscience. 5
Name Type Meaning Default value
events.spool.dir string path where to store events /GRID/common/spool
events.mode.dir string permissions used to create hash dir in event store 755
events.mode.file string permissions used to create event files 444
events.manage.enable boolean (yes|no) enable event management yes
events.max_pending integer maximum number of events in pending queue 100
events.incoming_delay integer minimum delay between event creation and treatment (in sec) 0
events.refresh_delay integer event routing table refresh freq (in sec) 30
nsinfo.refresh_delay integer namespace info refresh freq (in sec) 5

[server.inet] Directives

Name Type Meaning Default value
backlog integer (128<i<1024) TCP backlogs (overrides backlog directive in [General]) 256
port integer (1<i<65535) TCP socket port none

Example

gridagent.conf

[General]
user=admgrid
group=admgrid

backlog=4096

cluster_update_freq=2
service_check_freq=2

events.spool.dir=/DATA/$HOSTNAME/spool
events.mode.dir=755
events.mode.file=644
events.manage.enable=yes

unix_sock.gid=220
unix_sock.uid=120
unix_sock.mode=0660

[server.inet]
port=4000

gridagent.log4crc

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE log4c SYSTEM "">
<log4c>
    <config>
        <bufsize>2048</bufsize>
        <debug level="4"/>
        <nocleanup>0</nocleanup>
    </config>

    <category name="root" priority="info" appender="RC,common,common,gridagent"/>
    <appender name="RC,common,common,gridagent" type="syslog" layout="basic"/>
    <layout name="basic" type="basic"/>
</log4c>

Conscience

Conscience service is started by gridinit and has its own configuration file /GRID/$namespace/$HOSTNAME/conf/conscience.conf.

Usage

gridd CONFIG_PATH [LOG4C_PATH]

Configuration

Conscience configuration is done using a .ini-like config file parsed by glib (https://developer.gnome.org/glib/stable/glib-Key-value-file-parser.html).
It contains the following groups:

  • [General]
  • [Server.conscience]
  • [Plugin.stats]
  • [Plugin.fallback]
  • [Plugin.conscience]

[General] Directives

Name Type Meaning Default value
daemon boolean (true|false) demonize the service if true false
pidfile string location of the pid file /tmp/server.pid.XXXXXX
to_op integer socket operation timeout (in ms) 60 000
to_cnx integer socket connection timeout (in ms) 60 000

[Server.conscience] Directives

Name Type Meaning Default value
plugins string a comma separated list of modules to load none
listen string (ip:port) address and port to listen on none
min_workers integer minimum number of threads 1
max_workers integer maximum number of threads 100
min_spare_workers integer minimum number of idle thread 10
max_spare_workers integer maximum number of idle thread 100

[Plugin.stats] and [Plugin.fallback] Directives

Name Type Meaning Default value
path string location of the plugin (usually /usr/local/lib64/grid/msg_<plugin>.so) none

[Plugin.conscience] Directives

Name Type Meaning Default value
path string location of the plugin (usually /usr/local/lib64/grid/msg_conscience.so) none
param_namespace string the namespace name handled by this conscience none
param_chunk_size integer the chunk size limit (in bytes) none
param_serialize_srvinfo_cache boolean (true|false) activate serialization optimization with cache FALSE
param_serialize_srvinfo_tags boolean (true|false) cache service_info tags serialization TRUE
param_serialize_srvinfo_stats boolean (true|false) cache service_info stats serialization FALSE
param_events string location of the event routing configuration file none
param_storage_conf string location of the storage policy configuration file none
param_service.$SERVICE_TYPE.score_timeout integer the time without receiving stats after which the service score is set to 0 (in sec) 300
param_service.$SERVICE_TYPE.score_variation_bound integer the maximum raise of a growing service score 5
param_service.$SERVICE_TYPE.score_expr string the service score calculation formula 100

Example

conscience.conf

[General]
daemon=false
pidfile=/GRID/$namespace/$HOSTNAME/run/conscience-1.pid
to_op=300000
to_cnx=300000

[Server.conscience]
plugins=conscience,stats,fallback
listen=$port_conscience
min_workers=30
max_workers=200
min_spare_workers=30
max_spare_workers=100

[Plugin.stats]
path=/usr/local/lib64/grid/msg_stats.so

[Plugin.fallback]
path=/usr/local/lib64/grid/msg_fallback.so

[Plugin.conscience]
path=/usr/local/lib64/grid/msg_conscience.so
param_namespace=$namespace
param_chunk_size=10485760
param_meta0=$ip:$port_meta0
param_events=/GRID/$namespace/$HOSTNAME/conf/conscience-1.events

param_service.default.score_timeout=300
param_service.default.score_variation_bound=5
param_service.default.score_expr=100
param_service.meta1.score_timeout=300
param_service.meta1.score_variation_bound=5
param_service.meta1.score_expr=root(2,((num stat.cpu) * (num stat.req_idle)))
param_service.meta2.score_timeout=300
param_service.meta2.score_variation_bound=5
param_service.meta2.score_expr=((num stat.space)>=5) * root(2,((num stat.cpu)*(num stat.space)))
param_service.rawx.score_timeout=300
param_service.rawx.score_variation_bound=5
param_service.rawx.score_expr=((num stat.io)>=10) * ((num stat.space)>=3) * root(3,((num stat.cpu)*(num stat.io)*(num stat.space)))

conscience.log4crc

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE log4c SYSTEM "">
<log4c>
    <config>
        <bufsize>2048</bufsize>
        <debug level="4"/>
        <nocleanup>0</nocleanup>
    </config>

    <category name="root" priority="info" appender="RC,$namespace,$HOSTNAME,conscience"/>
    <appender name="RC,$namespace,$HOSTNAME,conscience" type="syslog" layout="basic"/>
    <layout name="basic" type="basic_r"/>
</log4c>

Meta0

This is the first-level directory service.
Meta0 is configured directly with options on command line, no specific configuration files is required, except its gridinit declaration in /GRID/$HOSTNAME/conf/gridinit.conf.d/meta0-1

Usage

meta0_server [OPTIONS] ... NAMESPACE_NAME DATADIR

Options:

-h Displays this help section
-d Detache then daemonize the gridinit
-q Quiet mode, suppress non-error output
-v[vv] verbose mode, this activates stderr traces (default FALSE)
-p location of a pid file
-s activates syslog traces (default FALSE) with the given identifier
-O Endpoint url to define listening addr and port

Example:

/usr/local/bin/meta0_server -v -p /GRID/$namespace/$HOSTNAME/run/meta0-1.pid -s RC,$namespace,$HOSTNAME,meta0-1 -O Endpoint=$ip:$meta0_port $namespace /DATA/$namespace/$HOSTNAME/meta0-1

Meta1

This is the second-level directory service.
Meta1 is configured directly with options on command line, no specific configuration files is required, except its gridinit declaration in /GRID/$HOSTNAME/conf/gridinit.conf.d/meta1-1

Usage

meta1_server [OPTIONS] ... NAMESPACE_NAME DATADIR

Options:

-h Displays this help section
-d Detache then daemonize the gridinit
-q Quiet mode, suppress non-error output
-v[vv] verbose mode, this activates stderr traces (default FALSE)
-p location of a pid file
-s activates syslog traces (default FALSE) with the given identifier
-O Endpoint url to define listening addr and port

Example:

/usr/local/bin/meta1_server -v -p /GRID/$namespace/$HOSTNAME/run/meta1-1.pid -s RC,$namespace,$HOSTNAME,meta1-1 -O Endpoint=$ip:$meta1_port $namespace /DATA/$namespace/$HOSTNAME/meta1-1

Meta2

This is the third-level directory service.
Meta2 is configured directly with options on command line, no specific configuration files is required, except its gridinit declaration in /GRID/$HOSTNAME/conf/gridinit.conf.d/meta2-1

Usage

meta2_server [OPTIONS] ... NAMESPACE_NAME DATADIR

Options:

-h Displays this help section
-d Detache then daemonize the gridinit
-q Quiet mode, suppress non-error output
-v[vv] verbose mode, this activates stderr traces (default FALSE)
-p location of a pid file
-s activates syslog traces (default FALSE) with the given identifier
-O Endpoint url to define listening addr and port
-O Tag Additional tag to associate to the service (multiple custom tags are supported)

Example:

/usr/local/bin/meta2_server -vv -p /GRID/$namespace/$HOSTNAME/run/meta2-1.pid -s RC,$namespace,$HOSTNAME,meta2-1 -O Endpoint=$meta2_ip:$meta2_port –O Tag=location=$datacenter.$room.$rack.$cluster.$server.$volume $namespace /DATA/$namespace/$HOSTNAME/meta2-1

RAWX

Rawx is the storage service and is implemented as an apache webdav repository module.
Redcurrant provides a wrapper called rawx-monitor to handle service registration sor Rawx.
Rawx configuration is done in httpd.conf format in /GRID/$namespace/$HOSTNAME/conf/rawx-1.conf

Configuration

The rawx webdav repository module has some specific configuration directives.

Directives

Name Type Meaning Default value
grid_hash_width integer number of characters in hash directory name 2
grid_hash_depth integer number of sub-directories in hash 2
grid_docroot string location of the datadir where to store chunks none
grid_namespace string name of the namespace the rawx belongs to none
grid_dir_run string location of the run dierctory where to store pid and shm files /var/run
grid_fsync boolean force filesystem sync on chunk storage FALSE

Example

rawx-1.conf

LoadModule dav_module        /usr/lib64/httpd/modules/mod_dav.so
LoadModule dav_rawx_module   /usr/lib64/httpd/modules/mod_dav_rawx.so
LoadModule mime_module          /usr/lib64/httpd/modules/mod_mime.so

Listen $ip:$raw_port
PidFile /GRID/$namespace/$HOSTNAME/run/rawx-1-httpd.pid
ServerRoot /GRID/$namespace/$HOSTNAME/core
ServerName localhost
ServerSignature Off
ServerTokens Prod
DocumentRoot /GRID/$namespace/$HOSTNAME/run
TypesConfig /etc/mime.types

User  admgrid
Group admgrid

ErrorLog /GRID/$namespace/$HOSTNAME/logs/rawx-1-httpd-errors.log
LogLevel debug

<IfModule mod_env.c>
SetEnv nokeepalive 1
SetEnv downgrade-1.0 1
SetEnv force-response-1.0 1
</IfModule>

<IfModule prefork.c>
MaxClients       150
StartServers       5
MinSpareServers    5
MaxSpareServers   10
</IfModule>

<IfModule worker.c>
StartServers           5
MaxClients           100
MinSpareThreads        5
MaxSpareThreads       25
ThreadsPerChild       10
MaxRequestsPerChild    0
</IfModule>

DavDepthInfinity Off

grid_hash_width 2
grid_hash_depth 2
grid_docroot /DATA/$namespace/$HOSTNAME/rawx-1
grid_namespace $namespace
grid_dir_run /GRID/$namespace/$HOSTNAME/run

<Directory />
DAV rawx
AllowOverride None
</Directory>

<VirtualHost $ip:$rawx_port>
# DO NOT REMOVE (even if empty) !
</VirtualHost>

Rawx-monitor

Usage

rawx-monitor CONFIG_PATH [LOG4C_PATH]

Configuration

Rawx-monitor configuration is done using a .ini-like config file parsed by glib (https://developer.gnome.org/glib/stable/glib-Key-value-file-parser.html).
It contains four groups:

  • [Default]
  • [Child]
  • [Service]
  • [Volume]

[Default] Directives

Name Type Meaning Default value
daemon boolean (true|false) demonize the service if true false
pidfile string location of the pid file /tmp/server.pid.XXXXXX none

[Child] Directives

Name Type Meaning Default value
command string command line to be forked by gridinit, with options none
respawn boolean restart rawx if it exits false
rlimit.stack_size integer The thread stack size to apply to forked processes (overrides value defined in gridinit.conf)
rlimit.core_size integer The core size limit to apply to forked processes (overrides value defined in gridinit.conf)
rlimit.max_files integer The maximum number of open fd to apply to forked processes (overrides value defined in gridinit.conf)

[Service] Directives

Name Type Meaning Default value
ns string the namespace name this rawx-monitor belongs to none
type string the service type name rawx
addr url the address and port the forked rawx listens to none
location string a location domain to be used by storage policies none

[Volume] Directives

Name Type Meaning Default value
docroot string the rawx datadir none

Example

rawx-monitor-1.conf

[Default]
daemon=false
pidfile=/GRID/$namespace/$HOSTNAME/run/rawx-1-monitor.pid

[Child]
command=/usr/sbin/httpd.worker -D FOREGROUND -f /GRID/$namespace/$HOSTNAME/conf/rawx-1-httpd.conf
respawn=true
rlimit.stack_size=1048576
rlimit.core_size=-1
rlimit.max_files=32768

[Service]
ns=$namespace
type=rawx
addr=$ip:$rawx_port
location=$datacenter.$room.$rack.$cluster.$server.$volume

[Volume]
Docroot=/DATA/$namespace/$HOSTNAME/rawx-1

RAINX

Rainx is the RAIN service and is implemented as an apache webdav repository module.
Rainx uses the same wrapper called rawx-monitor as Rawx to handle service registration (see configuration before).
Rainx configuration is done in httpd.conf format in /GRID/$namespace/$HOSTNAME/conf/rainx-1.conf

Configuration

The rainx webdav repository module has some specific configuration directives.

Directives

Name Type Meaning Default value
grid_namespace string name of the namespace the rainx belongs to none
grid_dir_run string location of the run dierctory where to store pid and shm files /var/run

SQLX

Sqlx is configured directly with options on command line, no specific configuration files is required, except its gridinit declaration in /GRID/$HOSTNAME/conf/gridinit.conf.d/sqlx-1

Usage

sqlx_server [OPTIONS] ... NAMESPACE_NAME DATADIR

Options:

-h Displays this help section
-d Detache then daemonize the gridinit
-q Quiet mode, suppress non-error output
-v[vv] verbose mode, this activates stderr traces (default FALSE)
-p location of a pid file
-s activates syslog traces (default FALSE) with the given identifier
-O Endpoint url to define listening addr and port

Example:

/usr/local/bin/sqlx_server -s 'RC,$namespace,$HOSTNAME,sqlx-1' -O Endpoint=$ip:$port_sqlx $namespace /DATA/$namespace/$HOSTNAME/sqlx-1/

Crawlers

Container purge

A container crawler is needed to handle deletion of chunks which were used in deleted, deduplicated or out of versions contents. This crawler is composed of two parts:

  • purge service which is unique per storage node and multi-namespace
  • a container crawler which is dedicated to a meta2 data volume

Each of them must be declared in gridinit in /GRID/$HOSTNAME/conf/gridinit.conf.d/purge_service and /GRID/$HOSTNAME/conf/gridinit.conf.d/purge_crawler-1

[Service.common-crawler-purge-1]
command=/usr/local/bin/action_purge_container_service -s "RC,common,common,purge_service"
enabled=true
on_die=respawn
group=common,crawler-purge
env.PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

For each meta2 running on a node, declare a crawler in gridinit

[Service.$namespace-$HOSTNAME-purge_crawler-1]
command=/usr/local/bin/crawler -Oinfinite=on -Otrip=trip_container -Oaction=action_purge_container -- -trip_container.s=/DATA/$namespace/$HOSTNAME/meta2-1 -action_purge_container.n=$namespace
enabled=true
on_die=respawn
group=common,crawler-purge
env.PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

Suffix -1 is used when several instance of the same service can be installed on one node

Container dedup

A deduplication crawler is available to deduplicate identical contents at container level. This crawler is composed of two parts:

  • a deduplication service which is unique per storage node and multi-namespace
  • a container crawler which is dedicated to a meta2 data volume

Each of them must be declared in gridinit in /GRID/$HOSTNAME/conf/gridinit.conf.d/dedup_service and /GRID/$HOSTNAME/conf/gridinit.conf.d/dedup_crawler-1

[Service.common-crawler-dedup-1]
command=/usr/local/bin/action_dedup_container_service -s "RC,common,common,crawler_dedup-1"
enabled=true
on_die=respawn
group=common,crawler-dedup
env.PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

For each meta2 running on a node, declare a crawler in gridinit

[Service.$namespace-$HOSTNAME-dedup_crawler-1]
command=/usr/local/bin/crawler -Oinfinite=on -Otrip=trip_container -Oaction=action_dedup_container -- -trip_container.s=/DATA/$namespace/$HOSTNAME/meta2-1 -action_dedup_container.n=$namespace
enabled=true
on_die=respawn
group=common,crawler-dedup
env.PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

Suffix -1 is used when several instance of the same service can be installed on one node


User Tools