(R)?ex the friendly automation framework

News

2023-08-05
Rex-1.14.3

Rex-1.14.3 is now available on CPAN. It contains bug fixes for local package installation, command existence checks, and Git tests.

2023-05-05
Rex-1.14.2

The Rex-1.14.2 release is now available on CPAN. It contains bug fixes for running local commands on Windows, cloning git repositories, and test suite fixes for the upcoming perl-5.38.0 release.

2023-03-17
Call for papers TPRC 2023

Dean Hamstead from the The Perl and Raku Foundation Marketing Committee has sent an invitation to present about Rex at TPRC 2023. I’m posting it here to increase visibility.

2023-03-05
Rex-1.14.1

The Rex-1.14.1 release is now available on CPAN. It contains bug fixes and documentation updates.

2023-02-05
Rex-1.14.0

The Rex-1.14.0 release is now available on CPAN. It contains improved Rexfile loading, documentation updates, and bumps the minimum required Perl version to 5.12.5.

Events

2021-03-08
Learning automation using Rex

Ferenc Erki (FErki) will be the guest of Gábor Szabó on the next Code Maven live stream to learn about automation using Rex. Register for the free event via Code Maven or Meetup, and join the discussion!

2020-03-05
Unexpected use cases with Rex

Unexpected use cases with Rex at the 22nd German Perl/Raku Workshop 2020 in Erlangen by Ferenc Erki (FErki).

2019-11-09
Rex & Friends

Rex & Friends talk at the Barcelona Perl & Friends 2019 by Ferenc Erki (FErki).

» Home » Docs » Rex book » Infrastructure » Example of a complete Rex code infrastructure

Example of a complete Rex code infrastructure

For maintainable and reusable code it is important to start with the right infrastructure choices and tools. In this chapter you will learn how to setup and design your project to achieve this.

You can find the code of this chapter on github.

Building services out of blocks with Rex

If you have a large environment with multiple services and a complex architecture it is important to design your code in a way that you can reuse most parts of it. To achieve this you need to follow some simple rules.

Source Control System

Currently Rex supports only *git* for module and service repositories. So in this example i will use git.

To manage your git repositories i suggest some tool like Gitprep or Gitlab

Modules

As mentioned above, you should separate your code. For better readability, usability and maintainability of your code this is important.

Modules must be generic. The modules are a bit like lego blocks. You can use lego blocks to build houses, ships, airplanes and much more. But the blocks are always the same.

Services

Services are a collection of modules. A service describe the system you want to build out of your module blocks.

CMDB

Rex has a default CMDB build upon YAML files. Store all the service relevant configuration options inside your YAML file. If you do this you can be sure that your code will work even if you change something in the CMDB.

Example frontend service

This is an example rex service.

meta.yml - Define dependencies

In the meta.yml file you can define some service information. The most important ones are the dependencies.

You can define dependencies to Rex modules and to Perl modules.

Name: Frontend Service
Description: The frontend service
Author: jan gehring 
License: Apache 2.0
Require:
  Rex::NTP::Base:
    git: https://github.com/krimdomu/rex-ntp-base.git
    branch: master
  Rex::OS::Base:
    git: https://github.com/krimdomu/rex-os-base.git
    branch: master
PerlRequire:
  - Moo

In this file you see that we define some dependencies to custom Rex modules located in git repositories. The rexify --resolve-deps command will read the meta.yml file and download all these dependencies into the lib directory.

With this in mind it is easy to have multiple environments with the same service pointing to different development branches.

Rexfile - The code

Every service has its own Rexfile.

Load all required modules

use Rex -feature => ['1.0'];
use Rex::CMDB;
use Rex::Test;
use Rex::Group::Lookup::INI;

These lines loads all required modules.

Load the server groups

groups_file "server.ini";

Load all server groups from the file server.ini.

Configure the CMDB

set cmdb => {
    type => "YAML",
    path => [
        "cmdb/{operatingsystem}/{hostname}.yml", "cmdb/{operatingsystem}/default.yml",
        "cmdb/{environment}/{hostname}.yml",     "cmdb/{environment}/default.yml",
        "cmdb/{hostname}.yml",                   "cmdb/default.yml",
    ],
};

Configure the CMDB. Here we define a custom search path. This will tell the CMDB to lookup the keys in the following order:

It is possible to use every Rex::Hardware variable inside the path.

Include all required Rex modules

include qw/
  Rex::OS::Base
  Rex::NTP::Base
  /;

Include all needed Rex modules. With include all the tasks inside these modules won't get displayed with rex -T.

The main task

task "setup",

The main task. If you don't define the servers (or groups) in the task definition you can use the cli paramter -G or -H.

task "setup", group => "frontend",

It is also possible to define the server or group to connect to.

make {
    # run setup() task of Rex::OS::Base module
    Rex::OS::Base::setup();

    # run setup() task of Rex::NTP::Base module
    Rex::NTP::Base::setup();
};

Inside the task we just call the tasks from the modules we have included above. All tasks can be called as a normal perl function, as long as the taskname doesn't conflict with other perl functions.

The last line

# the last line of a Rexfile
1;

The last line of a Rexfile is normaly a true value. This is not always needed, but it is safer to include it.

Test before ship

Since version 0.46 Rex comes with a integration test suite. It is based on Rex::Box and currently supports VirtualBox. With it you can spawn local Virtual Box VMs and test your tasks on it.

The tests are stored in the t directory.

Example t/base.t test file

use Rex::Test::Base;
use Rex -feature => ['1.0'];

Load the Rex::Test::Base framework and the Rex basic commands.

test {
    my $t = shift;
    $t->name("ubuntu test");

    # we will add more code here in a bit
};
1;

Create a new test named ubuntu test. For every test Rex will create a new vm.

  $t->base_vm("http://box.rexify.org/box/ubuntu-server-12.10-amd64.ova");
  $t->vm_auth( user => "root", password => "box" );

Define the url where to download the base VM image and the authentication.

  $t->run_task("setup");

Define which task to run on the VM.

  $t->has_package("vim");
  $t->has_package("ntp");
  $t->has_package("unzip");

  $t->has_file("/etc/ntp.conf");

  $t->has_service_running("ntp");

  $t->has_content( "/etc/passwd", qr{root:x:0:}ms );

  run "ls -l";
  $t->ok( $? == 0, "ls -l returns success." );

Run the tests. You can also use normal rex functions here.

At the end finish the tests with:

  $t->finish;

You can now run the tests with rex Test:run.

Getting the code

Now, when you all have commit to your git repository you can use the rexify command to download them to (for example) a central deployment server.

rexify --init=https://github.com/krimdomu/service-frontend.git

This will download everything into the folder service-frontend. The command also takes care of all dependencies and download them also into the lib folder.

Proudly powered by Perl and built with Statocles

GitHub repository and discussions / Chat on Matrix and IRC / Mailing list on Google Groups (retired: rex-users@freelists)

MetaCPAN / Twitter / StackShare / Server Fault   -.ô.-   Disclaimer