Wednesday, May 28, 2014

Building a Group-Office client with AngularJS and Bootstrap

To demonstrate how easy it is to build a client for Group-Office, I've created an AngularJS app. It demonstrates that the server side PHP code of Group-Office is in fact one big JSON API that can be talked to with a completely decoupled client.

It's a very simple interface that was just for a "Proof of concept" but here's a screenshot:

E-mail App in Angularjs Group-Office client

When we'll replace the current ExtJS 3 based user interface we'll use the same approach.

I've created a GitHub repository for the Group-Office client here:

https://github.com/mschering/angular-groupoffice

Note that this client will only work with Group-Office 6.0.3 which you can get here:

http://sourceforge.net/projects/group-office/files/6.0/


I hope you like this example! You can also integrate these kind of API calls into your website or another application. Let me know what you think about it!

 

Note about CORS

If you run the Group-Office server on a different domain than this client you'll need to enable CORS. Otherwise you'll get errors like:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote
resource at http://localhost/groupoffice. This can be fixed by moving the resource
to the same domain or enabling CORS.

You can read more about enabling CORS on Group-Office here:

https://www.group-office.com/wiki/CORS

Friday, May 16, 2014

Installing fast indexed search for IMAP e-mail

With Dovecot and apache SOLR it's possible to install very fast mail search. You can search through the entire message contents within fractions of a second.

I've used Ubuntu and Debian to set this up.

Tomcat SOLR


Install the tomcat solr server with the following command:

$ apt-get install solr-tomcat

Find the SOLR schema on page http://hg.dovecot.org/dovecot-2.2/file/tip/doc/solr-schema.xml

Make sure your dovecot version matches:

Dovecot 2.2: (Ubuntu 14.04)
http://hg.dovecot.org/dovecot-2.2/raw-file/e99cd21e1f92/doc/solr-schema.xml

Dovecot 2.1 (Debian Wheezy)
http://hg.dovecot.org/dovecot-2.1/raw-file/300a3a81c2cb/doc/solr-schema.xml

Download it like this:

$ wget http://hg.dovecot.org/dovecot-2.2/raw-file/e99cd21e1f92/doc/solr-schema.xml

Move the file to the right location:

$ mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak
$ mv solr-schema.xml /etc/solr/conf/schema.xml

$ service tomcat6 restart

Check if tomcat works by browsing to:

http://localhost:8080

Note: I use Proxmox with openvz and it was necessary to assign 2 cpu's to the openvz container to get tomcat to run properly!

Security note
The admin page is publicly accessible by default! So I made sure only local connections are allowed by adding this line between the <Host></Host> tag in /etc/tomcat6/server.xml

<Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="127.0.0.1"/>

Dovecot

Now setup Dovecot.

Install the plugin package:

$ apt-get install dovecot-solr

Then modify:

/etc/dovecot/conf.d/10-mail.conf:

...
# Space separated list of plugins to load for all services. Plugins specific to
# IMAP, LDA, etc. are added to this list in their own .conf files.
#mail_plugins =
mail_plugins = $mail_plugins fts fts_solr
...


/etc/dovecot/conf.d/90-plugin.conf:

...
plugin {
  fts = solr
  fts_solr = break-imap-search url=http://127.0.0.1:8080/solr/
}
...

The "break-imap-search" option will use Solr also for indexing TEXT and BODY searches. This makes your server non-IMAP-compliant, but it's what people want ;). This is always enabled in v2.1+.

Now when an IMAP client does a "SEARCH TEXT keyword command" you should see these log entries in /var/log/mail.log:

May 15 14:19:29 mail.example.com dovecot: indexer-worker(admin@intermesh.dev): Indexed 294 messages in INBOX

For more info read: http://wiki2.dovecot.org/Plugins/FTS/Solr

Enjoy the fast searches!

Friday, May 9, 2014

Setting up a Chat server with Prosody and Group-Office


A lot of customers requested a chat module for Group-Office. In my search for a
solution I stumbled upon a great XMPP/Jabber chat server called Prosody.

This is a short guide on how to connect this server with Group-Office.

Become root and install the Prosody XMPP server:

$ apt-get install prosody luarocks

Install lpy becuase it's required for the external external auth module:

$ luarocks install lpty

Download mod_auth_external for Prosody here:

https://hg.prosody.im/prosody-modules/raw-file/tip/mod_auth_external/mod_auth_external.lua

Put it in /usr/lib/prosody/modules/
eg.
cd /usr/lib/prosody/modules
wget https://prosody-modules.googlecode.com/hg/mod_auth_external/mod_auth_external.lua

Now create /usr/bin/GroupOfficeProsodyAuth.php. This script handles the
authentication requests from Prosody and it uses the Group-Office framework.
Note: If you installed Group-Office with the debian package it's already available in /usr/bin/GroupOfficeProsodyAuth.php
#!/usr/bin/php
<?php
/**
 * Group-Office
 * 
 * Copyright Intermesh BV. 
 * This file is part of Group-Office. You should have received a copy of the
 * Group-Office license along with Group-Office. See the file /LICENSE.TXT
 *
 * If you have questions write an e-mail to info@intermesh.nl
 * 
 * @license AGPL/Proprietary http://www.group-office.com/LICENSE.TXT
 * @link http://www.group-office.com
 * @copyright Copyright Intermesh BV
 * @version $Id: GroupOfficeProsodyAuth.php 17690 2014-06-17 07:02:34Z mschering $
 * @author Merijn Schering 
 * @package GO.base
 * 
 * test:
 * 
 * auth:username:groupoffice.domain.com:password
 */

// the logfile to which to write, should be writeable by the user which is running the server
$sLogFile  = "/var/log/prosody/prosody.log";

// set true to debug if needed
$bDebug  = false;

$oAuth = new exAuth($sLogFile, $bDebug);

class exAuth
{

 private $sLogFile;

 private $bDebug;

 private $rLogFile;

 private function _includeGO($server){
  //Try to detect a host based
  if(class_exists('GO')){
   return true;
  }
  if(file_exists('/etc/groupoffice/'.$server.'/config.php')){
   define('GO_CONFIG_FILE', '/etc/groupoffice/'.$server.'/config.php');
   require(GO_CONFIG_FILE);
   require_once($config['root_path'].'GO.php');
  }else
  {

   require_once('/usr/share/groupoffice/GO.php');
  }
  
  $this->writeDebugLog("Config file: ".GO::config()->get_config_file()." db: ".GO::config()->db_name." ".GO::config()->debug);
 }
 
 public function __construct($sLogFile, $bDebug)
 {

  $this->sLogFile  = $sLogFile;
  $this->bDebug  = $bDebug;
  
  $this->rLogFile = fopen($this->sLogFile, "a") or die("Error opening log file: ". $this->sLogFile);

  $this->writeLog("start");

  do {  
   $sData = substr(fgets(STDIN),0,-1);
   
   if($sData){
    
    $this->writeDebugLog("received data: ". $sData);
    $aCommand = explode(":", $sData);
    if (is_array($aCommand)){
     switch ($aCommand[0]){
      case "isuser":
       if (!isset($aCommand[1])){
        $this->writeLog("invalid isuser command, no username given");
        $this->failure();
       } else {
   
        $sUser = str_replace(array("%20", "(a)"), array(" ", "@"), $aCommand[1]);
        $this->writeDebugLog("checking isuser for ". $sUser);

        $this->_includeGO($aCommand[2]);
        $user = \GO\Base\Model\User::model()->findSingleByAttribute('username', $sUser);

        if ($user){        
         $this->writeLog("valid user: ". $sUser);
         
         $this->success();

        } else {
         $this->writeLog("invalid user: ". $sUser);
         $this->failure();
        }
        
        
       }
      break;
      case "auth":
       
       if (sizeof($aCommand) != 4){
        $this->writeLog("invalid auth command, data missing");
        $this->failure();
       } else {
        

        $sUser = str_replace(array("%20", "(a)"), array(" ", "@"), $aCommand[1]);
        $sPassword = $aCommand[3];
        $this->writeDebugLog("doing auth for ". $sUser);



        $this->_includeGO($aCommand[2]);

        $user = \GO::session()->login($sUser, $sPassword, false);

     
        if ($user) {
      
         $this->writeLog("authentificated user ". $sUser ." domain ". $aCommand[2]);
         $this->success();
        } else {
      
         $this->writeLog("authentification failed for user ". $sUser ." domain ". $aCommand[2]);
         $this->failure();
        }
       
       }
      break;
      case "setpass":
       // postavljanje zaporke, onemoguceno
       $this->writeLog("setpass command disabled");
       $this->failure();
      break;
      default:
       // ako je uhvaceno ista drugo
       $this->writeLog("unknown command ". $aCommand[0]);
       $this->failure();
      break;
     }
    } else {
     $this->writeDebugLog("invalid command string");
     $this->failure();
    }
   }

   unset($aCommand);
  } while (true);
 }

 public function __destruct()
 {
  $this->writeLog("stop");  
 }

 private function writeLog($sMessage)
 {
  if (is_resource($this->rLogFile)) {
   fwrite($this->rLogFile, date("r") ." [external_auth] ". $sMessage ."\n");
  }
 }

 private function writeDebugLog($sMessage)
 {
  if ($this->bDebug){
   $this->writeLog( date("r")." [external_auth_debug] ".$sMessage);
  }
 }
 
 
 private function success(){
  echo "1\n";
  if(class_exists('\GO')){
   \GO::unsetDbConnection();
  }

 }
 
 private function failure(){
  echo "0\n";
  if(class_exists('\GO')){
   \GO::unsetDbConnection();
  }
 }


 
}

Make this script executable:

$ chmod +x /usr/bin/GroupOfficeProsodyAuth.php

Edit /etc/prosody/prosody.cfg.lua and enable the "bosh" and "groups" module by removing the "--" in front of it. Then comment out authentication = "internal_plain":

and set:

 -- authentication = "internal_plain"
authentication = "external"
external_auth_command = "/usr/bin/GroupOfficeProsodyAuth.php"
external_auth_protocol = "generic"

cross_domain_bosh = true

Also set the groups file:

groups_file = "/home/groupoffice/chat/groups.txt";
Also setup your VirtualHost with the domain you run Group-Office on.
(See https://prosody.im/doc/configure for more info on configuring Prosody).

Example config file:
-- Prosody XMPP Server Configuration
--
-- Information on configuring Prosody can be found on our
-- website at http://prosody.im/doc/configure
--
-- Tip: You can check that the syntax of this file is correct
-- when you have finished by running: luac -p prosody.cfg.lua
-- If there are any errors, it will let you know what and where
-- they are, otherwise it will keep quiet.
--
-- Good luck, and happy Jabbering!


---------- Server-wide settings ----------
-- Settings in this section apply to the whole server and are the default settings
-- for any virtual hosts

-- This is a (by default, empty) list of accounts that are admins
-- for the server. Note that you must create the accounts separately
-- (see http://prosody.im/doc/creating_accounts for info)
-- Example: admins = { "user1@example.com", "user2@example.net" }
admins = { }

-- Enable use of libevent for better performance under high load
-- For more information see: http://prosody.im/doc/libevent
--use_libevent = true;

-- This is the list of modules Prosody will load on startup.
-- It looks for mod_modulename.lua in the plugins folder, so make sure that exists too.
-- Documentation on modules can be found at: http://prosody.im/doc/modules
modules_enabled = {

        -- Generally required
                "roster"; -- Allow users to have a roster. Recommended ;)
                "saslauth"; -- Authentication for clients and servers. Recommended if you want to log in.
                "tls"; -- Add support for secure TLS on c2s/s2s connections
                "dialback"; -- s2s dialback support
                "disco"; -- Service discovery
                "posix"; -- POSIX functionality, sends server to background, enables syslog, etc.

        -- Not essential, but recommended
                "private"; -- Private XML storage (for room bookmarks, etc.)
                "vcard"; -- Allow users to set vCards

        -- These are commented by default as they have a performance impact
                --"privacy"; -- Support privacy lists
                --"compression"; -- Stream compression (requires the lua-zlib package installed)

        -- Nice to have
                "version"; -- Replies to server version requests
                "uptime"; -- Report how long server has been running
                "time"; -- Let others know the time here on this server
                "ping"; -- Replies to XMPP pings with pongs
                "pep"; -- Enables users to publish their mood, activity, playing music and more
"register"; -- Allow users to register on this server using a client and change passwords

        -- Admin interfaces
                "admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands
                --"admin_telnet"; -- Opens telnet console interface on localhost port 5582

        -- HTTP modules
                "bosh"; -- Enable BOSH clients, aka "Jabber over HTTP"
                --"http_files"; -- Serve static files from a directory over HTTP

        -- Other specific functionality
                "groups"; -- Shared roster support
                --"announce"; -- Send announcement to all online users
                --"welcome"; -- Welcome users who register accounts
                --"watchregistrations"; -- Alert admins of registrations
                --"motd"; -- Send a message to users when they log in
                --"legacyauth"; -- Legacy authentication. Only used by some old clients and bots.
};

-- These modules are auto-loaded, but should you want
-- to disable them then uncomment them here:
modules_disabled = {
        -- "offline"; -- Store offline messages
        -- "c2s"; -- Handle client connections
        -- "s2s"; -- Handle server-to-server connections
};

-- Disable account creation by default, for security
-- For more information see http://prosody.im/doc/creating_accounts
allow_registration = false;

-- These are the SSL/TLS-related settings. If you don't want
-- to use SSL/TLS, you may comment or remove this
-- ssl = {
--      key = "/etc/prosody/certs/localhost.key";
--      certificate = "/etc/prosody/certs/localhost.crt";
--}

ssl = {
                key = "/etc/prosody/certs/group-office.com.key";
                certificate = "/etc/prosody/certs/group-office.com.crt";
        }

-- Force clients to use encrypted connections? This option will
-- prevent clients from authenticating unless they are using encryption.

c2s_require_encryption = false

-- Force certificate authentication for server-to-server connections?
-- This provides ideal security, but requires servers you communicate
-- with to support encryption AND present valid, trusted certificates.
-- NOTE: Your version of LuaSec must support certificate verification!
-- For more information see http://prosody.im/doc/s2s#security

s2s_secure_auth = false

-- Many servers don't support encryption or have invalid or self-signed
-- certificates. You can list domains here that will not be required to
-- authenticate using certificates. They will be authenticated using DNS.

--s2s_insecure_domains = { "gmail.com" }

-- Even if you leave s2s_secure_auth disabled, you can still require valid
-- certificates for some domains by specifying a list here.

--s2s_secure_domains = { "jabber.org" }

-- Required for init scripts and prosodyctl
pidfile = "/var/run/prosody/prosody.pid"

-- Select the authentication backend to use. The 'internal' providers
-- use Prosody's configured data storage to store the authentication data.
-- To allow Prosody to offer secure authentication mechanisms to clients, the
-- default provider stores passwords in plaintext. If you do not trust your
-- server please see http://prosody.im/doc/modules/mod_auth_internal_hashed
-- for information about using the hashed backend.

-- authentication = "internal_plain"

authentication = "external"
external_auth_command = "/usr/local/bin/GroupOfficeProsodyAuth.php"
external_auth_protocol = "generic"

--authentication = "anonymous"
cross_domain_bosh = true
-- Select the storage backend to use. By default Prosody uses flat files
-- in its configured data directory, but it also supports more backends
-- through modules. An "sql" backend is included by default, but requires
-- additional dependencies. See http://prosody.im/doc/storage for more info.

--storage = "sql" -- Default is "internal"

-- For the "sql" backend, you can uncomment *one* of the below to configure:
--sql = { driver = "SQLite3", database = "prosody.sqlite" } -- Default. 'database' is the filename.
--sql = { driver = "MySQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }
--sql = { driver = "PostgreSQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }

-- Logging configuration
-- For advanced logging see http://prosody.im/doc/logging
log = {
        info = "/var/log/prosody/prosody.log"; -- Change 'info' to 'debug' for verbose logging
        error = "/var/log/prosody/prosody.err";
        "*syslog";
}

----------- Virtual hosts -----------
-- You need to add a VirtualHost entry for each domain you wish Prosody to serve.
-- Settings under each VirtualHost entry apply *only* to that host.

--VirtualHost "localhost"

VirtualHost "intermesh.group-office.com"
        groups_file = "/home/govhosts/intermesh.group-office.com/data/chat/groups.txt";
        -- Assign this host a certificate for TLS, otherwise it would use the one
        -- set in the global section (if any).
        -- Note that old-style SSL on port 5223 only supports one certificate, and will always
        -- use the global one.
        ssl = {
                key = "/etc/prosody/certs/group-office.com.key";
                certificate = "/etc/prosody/certs/group-office.com.crt";
        }

------ Components ------
-- You can specify components to add hosts that provide special services,
-- like multi-user conferences, and transports.
-- For more information on components, see http://prosody.im/doc/components

---Set up a MUC (multi-user chat) room server on conference.example.com:
Component "conference.intermesh.group-office.com" "muc"

-- Set up a SOCKS5 bytestream proxy for server-proxied file transfers:
--Component "proxy.example.com" "proxy65"

---Set up an external component (default component port is 5347)
--
-- External components allow adding various services, such as gateways/
-- transports to other networks like ICQ, MSN and Yahoo. For more info
-- see: http://prosody.im/doc/components#adding_an_external_component
--
--Component "gateway.example.com"
--      component_secret = "password"

                                           
Restart the Prosody XMPP server:

$ service prosody restart

Now try to connect with your XMPP/Jabber client.

Use your Group-Office user, password and domain.

You can check /var/log/prosody/ for log files.

Make sure config.php of Group-Office is readable by the prosody user!

Now enable the chat module in Group-Office.

Troubleshooting

  1. Make sure config.php of Group-Office is readable by the prosody user
  2. Check that the following ports are accessible 5222,5223,5280,5281 (or forwarded if it's behind a router)
  3. Make sure PHP CLI is logging errors and check for PHP errors in syslog.
  4. To make SSL work on all clients I had to combine the ssl certificate and CA bundle

Update: I found the excellent program Converse.js that could be integrated in Group-Office 6.0 easily:

Additionally, I used the Prosody "groups" module to automatically deliver all Group-Office Chat users as contacts to the chat clients.