2014-02-09

Handbook for the DevOps for OpenERP Video Playlist

Introduction

Over the next week or two I plan to launch a 9 part series describing the environment I've set up to manage my customer's OpenERP sites "in the cloud".

DevOps for OpenERP is an extremely concise (under 30 mins), quick start guide to installing virtual machines in a cloud, provisioning them remotely with OpenERP instances, and then managing loading them with data from Google Spreadsheets via OpenERP's XMLRPC interface.


Tools Used

You will be introduced to the following Free and Open Source Software (FOSS) products:
Every tool used is freely available on the Internet. 

Note that I do pay iwStack.comby the hour, for hosting of Apache CloudStack, but CloudStack itself is a free download, if you already have a server on which to install it.


Contents

  1. Prepare cloud virtual machine in iwStack.com. (video)
    iwStack.com rents out an Apache CloudStack implementation. 
    CloudStack has the excellent advantage of allowing you fine grained control over root access virtual machines without any supplier lock-in at all.
    Here you will get a quick guide to creating a new virtual machine. 
  2. Install Ubuntu 13.10 in the VM (quick review, if you need it) (video)
    I've added this step in the blog just to "prove" that I use plain vanilla Ubuntu Saucy.  Since I promise to show you every you every single required step, installing Ubuntu is included.
  3. Create template from VM and 2nd VM from template.  (video)
    "Template" is Apache CloudStack's term for a ready to use VM, a.k.a virtual appliance. Here I show you how to get a second server up and running quickly from a "template" created from the first server.
  4. Install SaltStack master and minion.   (video)
    SaltStack is network provisioning for Python people.  OpenERP is written with Python as is much of Ubuntu's administrative stuff; the compatibility makes sense.
  5. Install OpenERP using a SaltStack state file. (video)
    I have created a SaltStack state file that installs PostgreSQL and OpenERP. 
    Here, I show you how to use it.
  6. Install and use GateOne; the HTTP terminal server (video)
    Normally, to access machines in your cloud you have to run an SSH client somewhere. GateOne is an SSH terminal server you access via HTTP from any browser.  You can authenticate yourself with your Google ID.  Run from Apache CloudStack, you can start up a GateOne server, do admin tasks on other servers using Salt, and then shut it down again. It's very hard to break into a server that is switched off! 
  7. Install And check GSpread (video)
    GSpread is a very easy to use, Python API for Google Spreadsheets.  I show you how to install, test and use it.
  8. Install and check OpenERP data pump (video)
    My data pump wraps GSpread and the OpenERP XMLRPC channels with a high-level data handling facility.  Typically you just need to get your data into correctly named columns in a Google Spreadsheet in order to be able to load it into OpenERP at high speeds.
  9. Demonstrate Google Spreadsheet to OpenERP data pump (video)
    I run through a few of the commands and data loading capabilities of the pump.

My quality rules 

  • I really hate tutorial videos that waste my time!  So, each video is 2 to 4 minutes: no fluff, no dead time.  You'll know in a minute or two whether the video is going to be useful to you.
  • Dark, blurry videos are stupid.  So, these are recorded as 1280x720 45fps H.264.
  • I dislike tutorial videos with audio!  So, there's no sound: you can run through at high speed, skimming text annotations and stopping where there's something you need. "Seeing is believing.", " A picture's worth a blah, blah blah"
  • I get mad if a tutorial video ends with something that doesn't work!   So, I start everything from original downloads and show you every installation step, complete. This ensures you have a good chance of getting the demonstrated results.  Be prepared to pause playback very frequently to get the full details. (If you have the right browser version you may have a playback speed control among the buttons on the bottom right.)
  • Then there're videos with a lot of typing and no where to get the text. So I provide a page with all the longer bits of text you'll need to type.


Usage Notes

  • Be prepared to pause the video frequently.  Accelerating the video is a bit like Dolby noise reduction -- boost things way up above the noise level, then rein back at the point of delivery.  You know what is noise for you.
  • You can remove any annotation that gets in your way.  Just click the little black ball at the top-right of the annotation's rectangle.
  • Annotations that have an arrow in a box on the right-hand side are links to text or URL's you'll need in order to follow along completing the associated action.

7 comments:

  1. Hi Martin,

    Thanks for sharing. This is really an interesting approach.

    I have 2 questions.

    a) Is there a way to automate setup of cloudstack's VM & template programtically? Same apply to salt and minion.

    Let's say I have an erp website (something like openerp online). When someone subscribe to an erp service e.g. mycompany.myopenerp.com (after making payment), it will trigger a script behind-the-scene which automate what's covered in your tutorial?

    Would this be possible with your suggested tools?

    If have to setup 1 by 1 even it's semi automated, that will require a lot of work? I can see the current openerp online is quite automated (no delays).

    b) As for your gdata pump and google spreadsheet, what if I have a custom openerp module which allow certain openerp instance to be installed? Example, CustomerA may need this module, but not for CustomerB and CustomerC. Can this feature be supported?

    Any help? Thanks.

    ReplyDelete
    Replies
    1. I really am delighted to hear from you. It's nice to know that someone has taken an interest, especially in view of the shocking amount of time it took me to do the videos.
      Q1: Is there a way to automate setup of cloudstack . . . ?
      A1: SaltStack recently added a module* for controlling CloudStack. It seems to connect and use the API*. Between those two you could certainly achieve what you intend.
      Q1: Support module installation with data pump?
      A1: If you look at the code of the pump you'll see that module installation is trivial : oerp.get('ir.modulemodule').button_immediate_install([idModule])
      I don't see the pump as the right tool for it, though. A salt stack module that used Sebastian Alix's *great* OpenERP XMLRPC library***, would make more sense.

      * http://docs.saltstack.com/en/latest/ref/clouds/all/salt.cloud.clouds.cloudstack.html
      ** https://cloudstack.apache.org/docs/api/apidocs-4.3/TOC_User.html
      *** http://pythonhosted.org//OERPLib/

      Delete
    2. Thanks Martin, let me digest through your links :)

      anyway correct me if i am wrong...

      i will have 1 salt master and multiple minions (assume 1 minion = 1 openerp installation).

      what if I have this:-
      - minion01 - openerp 7
      - minion02 - openerp 7
      - minion03 - openerp 8

      one more question, what if I want to use bzr pull instead of nightly approach? can this still be done? I am thinking bzr pull as a better approach to ensure the codes are always updated properly.

      What do you think? Thanks.

      Delete
    3. Salt Stack has powerful capabilities for specifying what versions to install on which set of minions.

      I am currently studying the whole question of backups and compatibility of code upgrades with past current and future data. It seems to be a can of worms.

      Delete
    4. Interesting :) Definitely have to explore more :)

      Do you think openerp online is using similar model (cloudstack, saltstack, etc)?

      Thanks.

      Delete
    5. I think it is probably done with software related to the RunBot, but I'm only guessing.

      Delete
    6. yeah :)

      by the way off topic, for openerp runbot, what are they using? buildbot?

      Delete