Posts Tagged ‘postgresql’

Updating Sequences automatically in Postgresql

Posted on: December 10th, 2010 by James Cluff 2 Comments

Updating Sequences Automatically in Postgresql

So about the second or third time you write something from scratch and it is a useful something and you’re thinking that next time you don’t want to write it again… well that’s a good time to record it somewhere. So recently we were doing a data migration project from version OpenERP 5.x to OpenERP 6.x.  We were doing this for our own company because version 6 of OpenERP hasn’t been released yet, but we want to use it as beta testers so we can contribute to, and help with development.  With an OpenERP Publishers Warranty, this won’t be an issue for users of OpenERP.  It is only an issue for us because we are testing on a non-released version, version 6.0. This issue can exist in any data conversion process using a Microsoft SQLSQL Server or ‘Sequel Server’, Oracle, IBM DB2, or MySQL and Postgresql. During the data conversion from one database to another database you often run into serial key, or sequence issues. OpenERP runs on Postgresql so this is written for Postgresql Sequence or Serial Key Issues. When you load data into a table the data includes ID’s from your old database, for example with a user id 100. There may have not been any id’s in the database you’re putting the data into so you have to set the sequencer to start at your highest ID value.  So if the highest id value in the table was 100 but your serial key is only 99 for example next time you add a record you will get an error message something like:
An error has occurred:

ERROR:  duplicate key value violates unique constraint "res_users_pkey"
The fix to this if you have only one is to simply do an alter sequence statement something like:
alter sequence mytable_seq restart with 100
However, if you have to do it like 100 times on a big database integration or simply want to have it as part of your script that moves data you could use a function to do that.  That function is the one I have written at least three times and the purpose of this post.  I hope it isn’t a standard function already in there that I don’t know about, if so, then maybe this post will encourage someone to tell me.  Anyway I am recording it here for my own re-use in the future. First you must install plpgsql in Linux you do this like this
createlang plpgsql database_name
You can do that from the command line as postgres or sudo as postgres into the database that you want to use plpgsql in. Then you can create this function
--drop function update_serial_key(character varying, character varying);
create function update_serial_key(tablename varchar, sequencename varchar)RETURNS integer AS $$

declare
 table_name alias for $1;
 c integer ;
Begin
EXECUTE 'select max(id) from '
|| table_name
 INTO c;

execute 'alter sequence '
||$2
||' restart with '
||c;
return c;
End
$$ LANGUAGE plpgsql;
Then at the end of your updates to a table, you can update the sequence by calling that function like this:
select update_serial_key('res_users', 'res_users_id_seq')
That is select update_serial_key (‘table_name’,'sequence_name’) So in my case at the end of migrating the users data from res_users of OpenERP 5 to OpenERP 6, I run that command and it automatically fixes the res_users_id_seq sequence to where it is suppose to be.  This is very handy for large data migration or upgrade projects when you might use it over and over and over again.

Installing OpenERP from source as a service

Posted on: November 3rd, 2010 by nuwan 2 Comments
Installing OpenERP especially for a new comer would be a little bit difficult at first, not because OpenERP itself is difficult to install, but if you follow the official documentation then probably you have chances to mess things up since the doc is not 100% up-to-date and it is somewhat misleading with all the comments from various users. Here I am going to guide you how to install OpenERP running as a service. The recent releases of debian based distros has OpenERP as deb package so one can easily install it with a synaptic package management system, but instead here I am going to show you how to install an OpenERP server and web client from source and then run it as a service.
Server : Ubuntu 10.04 (Lucid Lynx)
OpenERP version : 5.0.14
PostgreSQL version : 8.4
Python version : 2.6
You need administrative (sudo) privilages on the server. Login to the server as an administrative user (sudo).  We first need to install a PostgreSQL server if it is not already installed.
$ sudo apt-get install postgresql
Then we need to create a PostgreSQL database user for openerp. For doing this you need to switch to PostgreS user.
$ sudo su – postgres
$ createuser --createdb --username postgres --no-createrole --pwprompt openerp
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
$ exit
Now if you try to login to the database as this user, you will get an error.
$ psql -U openerp -W
$ psql: FATAL:  Ident authentication failed for user "openerp"
To fix this we need to change PostgrSQLl configuration so that it uses ident based authentication instead of password based authentication.
$ sudo vi /etc/postgresql/8.4/main/pg_hba.conf
change the line
#local   all         all                               ident
to
local   all         all                               md5
then it should be as following # Database administrative login by UNIX sockets local   all         postgres                          ident # TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD # “local” is for Unix domain socket connections only local   all         all                               md5 # IPv4 local connections: host    all         all         127.0.0.1/32          md5 # IPv6 local connections: host    all         all         ::1/128               md5 # IPv6 local connections: host    all         all         ::1/128               md5 Now download OpenERP server and web client to a convenient location. I downloaded them to the OpenERP directory created in my home directory.
$ mkdir ~/openerp
$ cd ~/openerp
$ wget  http://www.openerp.com/download/stable/source/openerp-server-5.0.14.tar.gz
$ wget http://www.openerp.com/download/stable/source/openerp-web-5.0.14.tar.gz
Extract the server and web archives.
$ tar zxvf  openerp-server-5.0.14.tar.gz
$ tar zxvf openerp-web-5.0.14.tar.gz

Installing OpenERP-server

To install the OpenERP server execute the setup script.
$ cd ~/openerp/openerp-server-5.0.14
$ sudo python setup.py install
The server is normally installed to /usr/local/lib/python2.6/dist-packages/openerp-server
Now you can run the server in console by executing the command.
$ openerp-server --db_user=openerp --db_password=<password>

Installing openerp-web

First you need to install the required libraries for the openerp-web.
$ sudo apt-get install  python-psycopg2 python-reportlab python-egenix-mxdatetime python-tz python-pychart python-pydot python-lxml python-vobject python-profiler
Also install python-dev and build-essential if they are not already installed.
$sudo apt-get install python-dev build-essential
$sudo apt-get install python-setuptools
Now change to the lib directory in the extracted openerp-web and run the populate.sh script. This will install all the dependencies required.
$ cd ~/openerp/openerp-web-5.0.14/lib
$ ./populate.sh
$ cd ..
Now you can run the web client by executing openerp-web in console.
$ openerp-web
You should be getting the following output.
[03/Nov/2010:10:11:35] ENGINE Bus STARTING
[03/Nov/2010:10:11:35] ENGINE Started monitor thread ‘_TimeoutMonitor’.
[03/Nov/2010:10:11:35] ENGINE Started monitor thread ‘Autoreloader’.
[03/Nov/2010:10:11:35] ENGINE Serving on 0.0.0.0:8080
[03/Nov/2010:10:11:35] ENGINE Bus STARTED

Running openerp-web as a service

Copy following openerp-web scripts from the installation directory in to specified locations.
$ sudo cp /usr/local/lib/python2.6/dist-packages/openerp_web-5.0.6-py2.6.egg/scripts/openerp-web /etc/init.d/
(This is the init script used to start/stop openerp-web)
$sudo cp /usr/local/lib/python2.6/dist-packages/openerp_web-5.0.6-py2.6.egg/config/openerp-web.cfg /etc/
(This is the openerp-web configuration file)
Grant execute permission to /etc/init.d/openerp-web
$sudo chmod +x /etc/init.d/openerp-web
Edit /etc/init.d/openerp-web file. In the file we need to provide the user who runs openerp. First we need to create new system user for this. I named mine as ‘openerp’.
$ useradd openerp
( this is the system user for running the server and web-client )
$ sudo vim /etc/init.d/openerp-web
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
DAEMON=/usr/local/bin/openerp-web
USER=”your system username”
Next edit /etc/openerp-web.cfg and specify the log files.
$ sudo gedit /etc/openerp-web.cfg
log.access_file = “/var/log/openerp-web/access.log”
log.error_file = “/var/log/openerp-web/error.log”
Lets create the log file directory and grant ownership to openerp user.
$ sudo mkdir /var/log/openerp-web/
$ sudo chown openerp /var/log/openerp-web/
Now run following command to start the OpenERP Web automatically on system startup (Debian/Ubuntu).
$ sudo update-rc.d openerp-web defaults
Now you can start the daemon like this:
$ sudo /etc/init.d/openerp-web start

Running openerp-server as a service

I could not find an init script for server in installation directory. So lets create a one.
$ sudo vim /etc/init.d/openerp-server
And enter the following script.
#!/bin/sh
### BEGIN INIT INFO
# Provides: openerp-server
# Required-Start: $syslog
# Required-Stop: $syslog
# Should-Start: $network
# Should-Stop: $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Enterprise Resource Management software
# Description: Open ERP is a complete ERP and CRM software.
### END INIT INFO
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
DAEMON=/usr/local/bin/openerp-server
NAME=openerp-server
DESC=openerp-server
USER=openerp
test -x ${DAEMON} || exit 0
set -e
case “${1}” in
start)
echo -n “Starting ${DESC}: “
start-stop-daemon –start –quiet –pidfile /var/run/${NAME}.pid \
–chuid ${USER} –background –make-pidfile \
–exec ${DAEMON} — –config=/etc/openerp-server.conf
echo “${NAME}.”
;;
stop)
echo -n “Stopping ${DESC}: “
start-stop-daemon –stop –quiet –pidfile /var/run/${NAME}.pid \
–oknodo
echo “${NAME}.”
;;
restart|force-reload)
echo -n “Restarting ${DESC}: “
start-stop-daemon –stop –quiet –pidfile /var/run/${NAME}.pid \
–oknodo
sleep 1
start-stop-daemon –start –quiet –pidfile /var/run/${NAME}.pid \
–chuid ${USER} –background –make-pidfile \
–exec ${DAEMON} — –config=/etc/openerp-server.conf
echo “${NAME}.”
;;
*)
N=/etc/init.d/${NAME}
echo “Usage: ${NAME} {start|stop|restart|force-reload}” >&2
exit 1
;;
esac
exit 0
Grant execute permissions to this script.
$ sudo chmod +x /etc/init.d/openerp-server
The following will make it run automatically on system startup.
$ sudo update-rc.d openerp-server defaults
Now we need to create a config file for the server. The server reads the config settings in the file on startup.
$ sudo vim /etc/openerp-server.conf
[options]
# Enable the debugging mode (default False).
verbose = False
debug_mode = False
# The file where the server pid will be stored (default False).
#pidfile = /var/run/openerp.pid
# The file where the server log will be stored (default False).
logfile = /var/log/openerp-server.log
# The unix account on behalf openerp is running.
process_user = openerp
# The IP address on which the server will bind.
# If empty, it will bind on all interfaces (default empty).
interface = localhost
# The TCP port on which the server will listen (default 8069).
#port = 8070
# Enable debug mode (default False).
debug_mode = False
# Launch server over https instead of http (default False).
secure = False
# Specify the SMTP server for sending email (default localhost).
smtp_server = localhost
# Specify the SMTP user for sending email (default False).
smtp_user = False
# Specify the SMTP password for sending email (default False).
smtp_password = False
# Specify the database name.
db_name = False
# Specify the database user name (default None).
db_user = openerp
# Specify the database password for db_user (default None).
db_password = <password_for_openerp_user>
# Specify the database host (default localhost).
db_host = localhost
# Specify the database port (default None).
db_port = 5432
# Specify the price accuracy.
#price_accuracy =
#Specify the addons path
addons-path = /usr/local/lib/python2.6/dist-packages/openerp-server/addons/
Lets create the server log file and grant the ownership to OpenERP user.
$ sudo touch /var/log/openerp-server.log
$ sudo chown openerp /var/log/openerp-server.log
That’s it. Now you can start the server by executing
$ sudo /etc/init.d/openerp-server start
Now from a web browser  navigate to OpenERP. You will get the OpenERP login screen.
http://yourdomain:8080

iBCScorp.com seeks Linux Administrator

Posted on: July 6th, 2010 by James Cluff 1 Comment

iBCScorp.com is seeking a Linux Systems Administrator for its Battaramulla, Sri Lanka office

This persons main responsibility will be to work with and to support staff in Sri Lanka, the US and customers in various locations.

Preferred experience for the Linux Systems Administrator

This persons should have 2-3 years or more experience in Linux systems administration using one of the popular Linux distributions preferably Ubuntu, RedHat, or Fedora. This person should have experience maintaining servers and doing most of the following tasks:
  • Creating scripts to manage the server maintenance events.
  • Creating Maintenance schedules and processes
  • Setting up monitoring of machines for performance, security and other events
  • Upgrading servers
  • DNS Setup and migration
  • Mail Server setup and maintenance
  • Setup and maintenance of at least one versioning system, GIT, Bazaar, CVS, SVN, etc.
  • familiarity with open source tools.

Linux Systems Administration duties

We are a development, integration company.  As such this person will be the go to person for all maintenance and administrative related tasks but as a team player should also be willing to learn how to program and work closely with developers to help understand their issues and to help make their jobs easier. Duties will include such issues as:
  • Setting up a postgresql or mysql database.
  • Tuning a server to get better performance from a database.
  • Installing mail servers or configuring mail servers.
  • Setting up a CMS or e-commerce system like Magento, Spree, Drupal, Joomla, OpenERP, SilverStripe, WordPress, etc.
  • Helping and supporting developers when they are stuck on administrative related issues.
  • Setting up and maintaining a Samba Server.
  • Setting up an Asterisk PBX.
  • Occasionally post articles related to our work.
  • be a team player and be willing to help on whatever else needs done be it helping with QA, development or whatever.

Key Characteristics for our Linux Systems Administrator.

This person should be someone who likes to learn how things work. They have to be able to solve problems on their own and troubleshoot problems which may not be immediately apparent. The Linux Systems Administrator must be someone who is willing to take responsibility to solve problems that others have not been able to solve, but also willing to take advise and to work as a team player drawing on experience from his/her comrades. This Systems Administrator should have good communication skills and listen so as to understand requirements. The Systems Administrator person should be continually trying to improve his/her skills. This Linux Administrator should pay close attention to detail and double check his/her work even though the tasks may seem simple or redundant so as not to frustrate production, security or delay things for our staff or customers.