Wednesday, May 28, 2014

Building a Group-Office client with AngularJS and Bootstrap

To demonstrate how easy it is to build a client for Group-Office, I've created an AngularJS app. It demonstrates that the server side PHP code of Group-Office is in fact one big JSON API that can be talked to with a completely decoupled client.

It's a very simple interface that was just for a "Proof of concept" but here's a screenshot:

E-mail App in Angularjs Group-Office client

When we'll replace the current ExtJS 3 based user interface we'll use the same approach.

I've created a GitHub repository for the Group-Office client here:

https://github.com/mschering/angular-groupoffice

Note that this client will only work with Group-Office 6.0.3 which you can get here:

http://sourceforge.net/projects/group-office/files/6.0/


I hope you like this example! You can also integrate these kind of API calls into your website or another application. Let me know what you think about it!

 

Note about CORS

If you run the Group-Office server on a different domain than this client you'll need to enable CORS. Otherwise you'll get errors like:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote
resource at http://localhost/groupoffice. This can be fixed by moving the resource
to the same domain or enabling CORS.

You can read more about enabling CORS on Group-Office here:

https://www.group-office.com/wiki/CORS

Friday, May 16, 2014

Installing fast indexed search for IMAP e-mail

With Dovecot and apache SOLR it's possible to install very fast mail search. You can search through the entire message contents within fractions of a second.

I've used Ubuntu and Debian to set this up.

Tomcat SOLR


Install the tomcat solr server with the following command:

$ apt-get install solr-tomcat

Find the SOLR schema on page http://hg.dovecot.org/dovecot-2.2/file/tip/doc/solr-schema.xml

Make sure your dovecot version matches.

Download it like this:

$ wget http://hg.dovecot.org/dovecot-2.2/raw-file/e99cd21e1f92/doc/solr-schema.xml

Move the file to the right location:

$ cp solr-schema.xml /etc/solr/conf/schema.xml

$ service tomcat6 restart

Check if tomcat works by browsing to:

http://localhost:8080

Dovecot

Now setup Dovecot. Modify:

/etc/dovecot/conf.d/10-mail.conf:

...
# Space separated list of plugins to load for all services. Plugins specific to
# IMAP, LDA, etc. are added to this list in their own .conf files.
#mail_plugins =
mail_plugins = $mail_plugins fts fts_solr
...


/etc/dovecot/conf.d/90-plugin.conf:

...
plugin {
  fts = solr
  fts_solr = break-imap-search url=http://localhost:8080/solr/
}
...

The "break-imap-search" option will use Solr also for indexing TEXT and BODY searches. This makes your server non-IMAP-compliant, but it's what people want ;). This is always enabled in v2.1+.

Now when an IMAP client does a "SEARCH TEXT keyword command" you should see these log entries in /var/log/mail.log:

May 15 14:19:29 mail.example.com dovecot: indexer-worker(admin@intermesh.dev): Indexed 294 messages in INBOX

Enjoy the fast searches!

Friday, May 9, 2014

Setting up a Chat server with Prosody and Group-Office


A lot of customers requested a chat module for Group-Office. In my search for a
solution I stumbled upon a great XMPP/Jabber chat server called Prosody.

It was very easy to connect this server with Group-Office. This is a short guide
on how to do it.

Become root and add the Prosody repository (replace squeeze with the debian or ubuntu version you are using):

$ echo deb http://packages.prosody.im/debian squeeze main | tee -a /etc/apt/sources.list

Add their key file:

$ wget https://prosody.im/files/prosody-debian-packages.key -O- | apt-key add -

Update APT sources:

$ apt-get update

No install Prosody XMPP server:

$ apt-get install prosody luarocks

Install lpy becuase it's required for the external external auth module:

$ luarocks install lpty

Download mod_auth_external for Prosody here:

https://code.google.com/p/prosody-modules/source/browse/mod_auth_external/mod_auth_external.lua

Put it in /usr/lib/prosody/modules/

Now create /usr/local/bin/GroupOfficeProsodyAuth.php. This script handles the
authentication requests from Prosody and it uses the Group-Office framework.

#!/usr/bin/php

 * @package GO.base
 */

// the logfile to which to write, should be writeable by the user 
// that is running the server
$sLogFile  = "/var/log/prosody/prosody.log";

// set true to debug if needed
$bDebug  = false;

$oAuth = new exAuth($sLogFile, $bDebug);

class exAuth
{

 private $sLogFile;

 private $bDebug;

 private $rLogFile;

 private function _includeGO($server){
  //Try to detect the config file.
        if(file_exists('/etc/groupoffice/'.$server.'/config.php')){
                define('GO_CONFIG_FILE', '/etc/groupoffice/'.$server.'/config.php');
                require(GO_CONFIG_FILE);
                require_once($config['root_path'].'GO.php');
        }else
        {

                require_once('/usr/share/groupoffice/GO.php');
        }

  
  $this->writeDebugLog("Config file: ".GO::config()->get_config_file());
 }
 
 public function __construct($sLogFile, $bDebug)
 {

  $this->sLogFile  = $sLogFile;
  $this->bDebug  = $bDebug;
  
  $this->rLogFile = fopen($this->sLogFile, "a") or die("Error opening log file: ". $this->sLogFile);

  $this->writeLog("start");

  do {  
   $sData = substr(fgets(STDIN),0,-1);
   
   if($sData){
    
    $this->writeDebugLog("received data: ". $sData);
    $aCommand = explode(":", $sData);
    if (is_array($aCommand)){
     switch ($aCommand[0]){
      case "isuser":
       if (!isset($aCommand[1])){
        $this->writeLog("invalid isuser command, no username given");
        $this->failure();
       } else {
   
        $sUser = str_replace(array("%20", "(a)"), array(" ", "@"), $aCommand[1]);
        $this->writeDebugLog("checking isuser for ". $sUser);

        $this->_includeGO($aCommand[2]);
        $user = \GO\Base\Model\User::model()->findSingleByAttribute('username', $sUser);

        if ($user){        
         $this->writeLog("valid user: ". $sUser);
         
         $this->success();

        } else {
         $this->writeLog("invalid user: ". $sUser);
         $this->failure();
        }
        
        
       }
      break;
      case "auth":
       
       if (sizeof($aCommand) != 4){
        $this->writeLog("invalid auth command, data missing");
        $this->failure();
       } else {
        

        $sUser = str_replace(array("%20", "(a)"), array(" ", "@"), $aCommand[1]);
        $sPassword = $aCommand[3];
        $this->writeDebugLog("doing auth for ". $sUser." with pass ".$sPassword);



        $this->_includeGO($aCommand[2]);

        $user = \GO::session()->login($sUser, $sPassword);

     
        if ($user) {
      
         $this->writeLog("authentificated user ". $sUser ." domain ". $aCommand[2]);
         $this->success();
        } else {
      
         $this->writeLog("authentification failed for user ". $sUser .
          " domain ". $aCommand[2]);
         $this->failure();
        }
       
       }
      break;
      case "setpass":
       $this->writeLog("setpass command disabled");
       $this->failure();
      break;
      default:
       $this->writeLog("unknown command ". $aCommand[0]);
       $this->failure();
      break;
     }
    } else {
     $this->writeDebugLog("invalid command string");
     $this->failure();
    }
   }

   unset($aCommand);
  } while (true);
 }

 public function __destruct()
 {
  $this->writeLog("stop");  
 }

 private function writeLog($sMessage)
 {
  if (is_resource($this->rLogFile)) {
   fwrite($this->rLogFile, date("r") ." [external_auth] ". $sMessage ."\n");
  }
 }

 private function writeDebugLog($sMessage)
 {
  if ($this->bDebug){
   $this->writeLog( date("r")." [external_auth_debug] ".$sMessage);
  }
 }
 
 
 private function success(){
  echo "1\n";
 }
 
 private function failure(){
  echo "0\n";
 }


 
}


Make this script executable:

$ chmod +x /usr/local/bin/GroupOfficeProsodyAuth.php

Edit /etc/prosody/prosody.cfg.lua and set:

authentication = "external"
external_auth_command = "/usr/local/bin/GroupOfficeProsodyAuth.php"
external_auth_protocol = "generic"

Also setup your VirtualHost with the domain you run Group-Office on.
(See https://prosody.im/doc/configure for more info on configuring Prosody).

Restart the Prosody XMPP server:

$ service prosody restart

Now try to connect with your XMPP/Jabber client.

Use your Group-Office user, password and domain.

You can check /var/log/prosody/ for log files.

Now all we need is an XMPP web interface for Group-Office!

Update: I found the excellent program Converse.js that could be integrated in Group-Office 6.0 easily:

Additionally, I used the Prosody "groups" module to automatically deliver all Group-Office Chat users as contacts to the chat clients.

Wednesday, April 9, 2014

Migrating email to your own mail server

To move your email to a new mail server you'll need setup the new server with all the existing accounts and copy the mailboxes.

Since this task is often at hand we had to find an efficient way to do this quickly so we wrote a script that would:
  1. Create all the existing mailboxes on the new server.
  2. Sync all the mail from the old server to the new one.
  3. Create Users that have access to this mailboxes.


For this you will need:
  • Group Office
  • ImapSync
  • The script at the bottom of this article


ImapSync packages are available for Debian here: http://archive.debian.net/etch/net/imapsync

Step 1: Setup your new mail server (together with Group Office)

The new email server needs to be installed first.
We have created a package for Debian/Ubuntu that would install and configure a postfix/dovecot mail server (almost) automatically

See our Wiki page for a link to these apt-get packages that would install and configure Group Office, Postfix (MTA for SMTP) and Dovecote (MDA for IMAP)
https://www.group-office.com/wiki/Installing_on_Debian_or_Ubuntu

When Group Office and the Mail server are installed open it in the browser and install the Postfixadmin module by going to Start menu (top right corner) -> Modules -> Install (top left) and choice Postfix Admin. This module lets you configure virtual mail domain and mailboxes. But don't create then yet. This will be done automatically in the next step.

Step 2: Create mailboxes on your new server

First you'll need a list of the existing mailboxes on the source server you want to fetch the mail from. You'll need to following details:
  1. Email
  2. Password
  3. First name
  4. Middle name
  5. Last name
The above information will be used to create mailboxes in the new server (with the same username and password) and the fetch the mail from the old server. If you don't know the password you'll have to change it because you won't be able to read to mailbox without it.

Put the mailbox information into a mailboxes.csv file like this.
michael@group-office.com,S3cRetPassWD,Michael,de,Hart
merijn@group-office.com,S3cRetPassWD,Merijn,,Schering
wesley@group-office.com,S3cRetPassWD,Wesley,,Smits
wilmar@group-office.com,S3cRetPassWD,Wilmar,van,Beusekom


Step 3: Running the mailbox migration script

After creating the CSV file that contain the information for the mailboxes The script at the bottom of this article will do the rest of the work.

Run it on the command line (not in the browser)

$ php syncmail.php

The script will need to know the path the you mailboxes.csv file. It it is in the same directory press enter. Otherwise enter the path to this file first. It the file is found it will show you a list of mailboxes it is going to migrate and asks you the following question:
- Do you want to create to following mailboxes? [y/N]:

Type 'y' and press enter

This won't take long and the following question will be asked:
- Fill mailboxes from another imap host? [y/N]:

Before you continue confirm the mailboxes are created on your new mail server by logging in to Group Office and open the Postfix admin module you've installed in Step 1

Type 'y' and press enter

ImapSync will need to know the target and source server address to fetch from and copy to. You'll be prompted for this information:
Enter IMAP host to fetch from: mail.oldserver.com
Port number [143]:
Enter IMAP host to copy to: [localhost]:
Port number [143]:

When ImapSync is installed correctly the script will start syncing all the created mailboxes (this could take a while)

Step 4: Creating Group Office user accounts for each mailbox (optional)

The last part the script takes care of is creating Group Office users that may access these mailboxes. If this is already setup our you would rather do this manualy just hit 'n'

Do you want to add/create users with these mail accounts in this Group Office installation [y/N]:

When entering 'y' the script will create Group Office users accounts. The username will be what is before the @ of the mailbox and the password will be the same.

<?php
/*
 * The MIT License (MIT)
 * 
 * Copyright (c) 2014 Intermesh BV 
 * 
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 * 
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */

/**
 * Script askes the following questions:
 * 
 * - Do you want to create to following mailboxes? [y/N]: 
 * - Fill mailboxes from another imap host? [y/N]:  
 * - Do you want to add/create users with these mailaccounts in this GroupOffice installation [y/N]:
 * 
 * RUN ON CLI AS ROOT !!
 * YOU MOST LIKELY WANT TO CHANGE THE PATH TO GO AND CONFIG.PHP
 * 
 * CSV should look like this:
 * email,password,firstname,middlename,lastname\n
 * 
 * @author Michael de Hart 
 * @copyright Copyright Intermesh BV.
 */

//first argument is the path to config.php
$conf_path = '/etc/groupoffice/config.php';

define('GO_CONFIG_FILE',$conf_path);
define('NOLOG', true); // stop groupoffice from logging. 
require_once('/usr/share/groupoffice/GO.php');
GO::session()->runAsRoot();

$syncer = new ImapSync();
$syncer->start();

class ImapSync {

 protected $imap_sync = '/usr/bin/imapsync'; //binary
 protected $domain = 'superfun.nl';
 protected $quota = 524288; //quota in kb = (512MB)
 protected $source = array(
  'host'=>'',
  'port'=>'143',
 );
 protected $target = array(
  'host'=>'localhost',
  'port'=>'143',
  'smtp_host'=>'localhost',
  'smtp_port'=>'25',
 );
 // CSV Config
 protected $csv = array(
  'path'=>'mailboxes.csv',
  'delimiter'=>",",
  'enclosure'=>'"'
 );
 // Column configuration for Mailbox CSV file
 protected $col = array(
  'email'=>0,
  'password'=>1,
  'first_name'=>2,
  'middle_name'=>3,
  'last_name'=>4,
 );
 protected $log_file = 'imapsync';
 private $records = array();

 public function start() {

  try {
   $handle = fopen("php://stdin","r");
   
   echo "This script will create mailboxes and users based on a CSV file?\n"
   . "Enter the path to the CSV file to continue: [".$this->csv['path']."]: ";
   $line = trim(fgets($handle));
   if(!empty($line)){
    $this->csv['path'] = $line;
   }
   $this->records = $this->loadCsv();
   
   if(GO::modules()->isInstalled('postfixadmin')) {
    echo implode(array_keys($this->records),"\n")."\n".
    "Do you want to create to following mailboxes? [y/N]: ";
    $line = fgets($handle);
    if(trim($line) == 'y'){
     echo "\nCreating mailboxes...\n";
     $this->createMailBoxes();
    }

    echo "Fill mailboxes from another imap host? [y/N]: ";
    $line = fgets($handle);
    if(trim($line) == 'y'){
     $this->configureSync($handle);
     echo "\nSyncing IMAP mailboxes...\n";
     foreach($this->records as $record) {
      $this->syncImap($record);
     }
    }
   }

   echo "Do you want to add/create users with these mailaccounts in this GroupOffice installation [y/N]: ";
   $line = trim(fgets($handle));
   if($line == 'y'){
    echo "\nEnter SMTP host for the accounts [".$this->target['smtp_host']."]: ";
    $line = trim(fgets($handle));
    if(!empty($line))
     $this->target['smtp_host'] = $line;
    echo "\nPort number [".$this->target['smtp_port']."]: ";
    $line = trim(fgets($handle));
    if(!empty($line))
     $this->target['smtp_port'] = $line;
    echo "\nAdding/creating users and accounts...\n";
    $this->createUsersWithMailAccounts();
   }
   echo "All done!\n";

  } catch(Exception $e) {
   echo $e->getMessage()."\n";
  }
 }
 
 private function configureSync($handle) {
  echo "\nEnter IMAP host to fetch from: ";
  $line = trim(fgets($handle));
  if(!empty($line))
   $this->source['host'] = $line;
  echo "\nPort number [".$this->source['port']."]: ";
  $line = trim(fgets($handle));
  if(!empty($line))
   $this->source['port'] = $line;
  echo "\nEnter IMAP host to copy to [".$this->target['host']."]: ";
  $line = trim(fgets($handle));
  if(!empty($line))
   $this->target['host'] = $line;
  echo "\nPort number [".$this->target['port']."]: ";
  $line = trim(fgets($handle));
  if(!empty($line))
   $this->target['port'] = $line;
 }
 
 //Read CSV file to an array
 private function loadCsv() {
  $records=array();
  $fp = @fopen($this->csv['path'], "r");
  if(!$fp)
   die("Can't find: " . $this->csv['path']."\n");
  while ($record = fgetcsv($fp, 4096, $this->csv['delimiter'], $this->csv['enclosure'])) {
   $assocRecord = array();
   foreach($this->col as $key => $number)
    $assocRecord[$key] = $record[$number];
   $records[$assocRecord['email']] = $assocRecord;
  }
  //Get domain from last email account
  if(isset($records[0])) {
   $this->domain = end(explode('@',$records[0]['email'],2));
  }
  return $records;
 }

 //Fetch all Mail and copy to new IMAP server using ImapSync
 private function syncImap($record){
  echo "Syncing " . $record['email'] . "...\n\n";
  $cmd = $this->imap_sync.' --syncinternaldates --authmech1 LOGIN --authmech2 LOGIN ' .
   '--host1="'.$this->source['host'].'" --user1="'.$record['email'].'" --password1="'.$record['password'].'" ' .
   '--host2="'.$this->target['host'].'" --user2="'.$record['email'].'" --password2="'.$record['password'].'" '.
   '--subscribe --allowsizemismatch --nofoldersizes '.
   '--sep1 / --sep2 . --regextrans2 "s,/,_,g"'; // --folder "INBOX"
  if(!empty($this->log_file))
   $cmd .= ' > '.__DIR__.'/'.$this->log_file.'_'.date('Y-m-d\ H:i:s').'.log';
  system($cmd);
 }

 // Create GroupOffice Mailboxes
 private function createMailBoxes() {
  $domain = GO_Postfixadmin_Model_Domain::model()->findSingleByAttribute('domain',$this->domain);
  if(empty($domain)) {
   $domain = new GO_Postfixadmin_Model_Domain();
   $domain->transport='virtual';
   $domain->active='1';
   $domain->domain=$this->domain;
   $domain->default_quota=$this->quota;
   $domain->user_id=1;
   if(!$domain->save())
    throw new Exception('Error while saving mailbox model: '.$record['email']."\n".implode("\n",$domain->getValidationErrors()));
  }
  foreach($this->records as $record) {
   $username = current(explode("@", $record['email']));
   $mailbox = GO_Postfixadmin_Model_Mailbox::model()->findSingleByAttribute('username',$username);
   if(!$mailbox){
    $mailbox = new GO_Postfixadmin_Model_Mailbox();
    $mailbox->domain_id = $domain->id;
    $mailbox->username = $record['email']; //$username;
    $mailbox->quota = $domain->default_quota;
   }
   $mailbox->name = $record['email'];
   $mailbox->password = $record['password'];
   echo "Saving mailbox ".$mailbox->username." with quota ".($mailbox->quota/1024)." MB...\n";
   if(!$mailbox->save())
    throw new Exception('Error while saving mailbox model: '.$record['email']."\n".implode("\n",$mailbox->getValidationErrors())); 
  }
  echo "All mailboxes are created, time to fetch from source mail host...\n";
  
 }

 private function createUsersWithMailAccounts() {
  foreach($this->records as $record) {
   $username = current(explode("@", $record['email']));

   // Check if the user exists in Group-Office and if it doesn't create it.
   $user = GO_Base_Model_User::model()->findSingleByAttribute('username',$username);
   if(!$user){
    $user = new GO_Base_Model_User();
    $user->first_name = $record['first_name'];
    $user->middle_name = $record['middle_name'];
    $user->last_name = $record['last_name'];
    $user->email = $record['email'];
    $user->enabled = 1;
   }
   $user->username = $username;
   $user->password = $record['password'];
   if(!$user->save())
    throw new Exception('Error while saving user: '.$username."\n".implode("\n",$user->getValidationErrors()));

   // Create an e-mail account for the user
   $account = GO_Email_Model_Account::model()->findSingleByAttributes(array(
    'username'=>$record['email'],
    'user_id'=>$user->id
   ));
   if(!$account) {
    $account = new GO_Email_Model_Account();
    $account->user_id = $user->id;
    $account->mbroot = '';
    $account->use_ssl = 0;
    $account->type = 'imap';
    $account->smtp_encryption = '';
    $account->smtp_username = '';
    $account->smtp_password = '';
    $account->name = $user->email;
    $account->email = $user->email;
   }
   $account->host = $this->target['host'];
   $account->port = $this->target['post'];
   $account->smtp_host = $this->target['smtp_host'];
   $account->smtp_port = $this->target['smtp_port'];
   $account->username = $record['email'];
   $account->password = $record['password'];
   if(!$account->save())
    throw new Exception('Error while saving account: '.$username."\n".implode("\n",$account->getValidationErrors()));

   echo $username."\n";
  }
 }

}

Wednesday, March 26, 2014

Automatically upload a screenshot from the clipboard on paste using Javascript in Google Chrome, Internet Explorer 11 and Firefox

I was working on a new documentation site and had to make a lot of screenshots. I thought it would be useful to be able to hit print screen and then paste the image directly into Group-Office using the CTRL+V shortcut. Luckily with HTML5 you can do that!

With Google Chrome and Internet Explorer 11 you can use the Clipboard API. I created a "Paster" class that you can pass a DOM object and a callback function. So when you paste an image into that object the file is uploaded to a PHP script and then the callback function is called with the response.

In this working example it will call the PHP page itself and then put the response in the textarea. It should work on any PHP enabled web server.

You can add an onpaste event to any DOM element in Google Chrome. Then the "Paster" loops through all clipboard items and checks if there's an image:


Paster.prototype.handlePaste = function(paster, e) {

 //don't do this twice
 if (paster.processing) {
  return;
 }

 //loop through all clipBoardData items and upload it if it's a file.
 for (var i = 0; i < e.clipboardData.items.length; i++) {
  var item = e.clipboardData.items[i];
  if (item.kind === "file") {

   paster.processing = true;
   e.preventDefault();
   paster.uploadFile(paster, item.getAsFile());
  }
 }
};

Then it uploads the file using an "XMLHttpRequest" call. When the upload is finished the callback function is called.

Paster.prototype.uploadFile = function(paster, file) {

 var xhr = new XMLHttpRequest();

 //progress logging
 xhr.upload.onprogress = function(e) {
  var percentComplete = (e.loaded / e.total) * 100;

  console.log(percentComplete);
 };

 //called when finished
 xhr.onload = function() {
  if (xhr.status === 200) {
   alert("Sucess! Upload completed. PHP response will be put in the textarea.");
  } else {
   alert("Error! Upload failed");
  }

  paster.processing = false;
 };

 //error handling
 xhr.onerror = function() {
  alert("Error! Upload failed. Can not connect to server.");
 };

 //trigger a callback when it's successful
 xhr.onreadystatechange = function()
 {
  if (xhr.readyState === 4 && xhr.status === 200)
  {
   if (paster.callback) {
    paster.callback.call(paster.scope || paster, paster, xhr);
   }
  }
 };


 //prompt for the filename
 var filename = prompt("Please enter the file name", "Pasted image");

 if (filename) {

  //upload the file
  xhr.open("POST", "", {
   filename: filename,
   filetype: file.type
  });
  var formData = new FormData();
  formData.append("pastedFile", file);
  xhr.send(formData);
 }
}

Firefox

So far it was pretty simple! But then I started testing this in Firefox. Unfortunately it didn't work because Firefox doesn't support files in the clipboard. I came up with a work around though.

You can paste images in a content editable div in Firefox. The blob data is then available in the image source tag: <img src="data:image/png;base64,DATA" />. So in my workaround I create a hidden content editable div. When ever the textarea blurs, it will focus this hidden element. So when you click out of the textarea and then hit CTRL+V, the image is pasted in the hidden div. The paster then grabs that image source and creates a file blob object. This file can be uploaded just like in Google Chrome. Not as user friendly because you have to paste outside the textarea but it works:

Paster.prototype.init=function() {

 var paster = this;

 if (window.Clipboard) {
  //IE11, Chrome, Safari
  this.pasteEl.onpaste=function(e){
   paster.handlePaste(paster, e);
  };
 } else
 {
  //On Firefox use the contenteditable div hack
  this.canvas = document.createElement('canvas');
  this.pasteCatcher = document.createElement("div");
  this.pasteCatcher.setAttribute("id", "paste_ff");
  this.pasteCatcher.setAttribute("contenteditable", "");
  this.pasteCatcher.style.cssText = 'opacity:0;position:fixed;top:0px;left:0px;';
  this.pasteCatcher.style.marginLeft = "-20px";
  document.body.appendChild(this.pasteCatcher);


  this.pasteEl.onblur=function(e) {
   paster.pasteCatcher.focus();
  };

  this.pasteCatcher.onpaste=function(e) {

   paster.findImageEl(paster);
  };
 }
};

 
Paster.prototype.dataURItoBlob=function(dataURI, callback) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this

 console.log(dataURI);

 var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
 var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]

// write the bytes of the string to an ArrayBuffer
 var ab = new ArrayBuffer(byteString.length);
 var ia = new Uint8Array(ab);
 for (var i = 0; i < byteString.length; i++) {
  ia[i] = byteString.charCodeAt(i);
 }

// write the ArrayBuffer to a blob, and you're done

 return new Blob([ia], {type: mimeString});
};

Paster.prototype.findImageEl = function(paster) {

 if (paster.pasteCatcher.children.length > 0) {

  var dataURI = paster.pasteCatcher.firstElementChild.src;
  if (dataURI) {
   if (dataURI.indexOf('base64') === -1) {
    alert("Sorry, with Firefox you can only paste local screenshots and files. Use Chrome or IE11 if you need paster feature.");
    return;
   }

   var file = paster.dataURItoBlob(dataURI);
   paster.uploadFile(paster, file);
  }

  paster.pasteCatcher.innerHTML = '';

 } else
 {
  setTimeout(function() {
   paster.findImageEl(paster);
  }, 100);
 }
};


Complete example

I hope this is useful for you! It sure is useful for the Group-Office Content Management System and E-mail module! Here's the fully functional example that should work on any php server. Save it to pasteupload.php:


<?php
/**
 *
 * The MIT License (MIT)
 *
 * Copyright (c) 2014 Intermesh BV 
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 *
 */

if($_SERVER['REQUEST_METHOD']=='POST'){
 
 var_dump($_FILES);
 
 exit();
}
?>
<html>
<head>
<script type="text/javascript">

//Define paster object with contructor
var Paster = function(config) {

 for(var key in config){
  this[key]=config[key];
 }
 
 this.init();
};

Paster.prototype.pasteEl=null;

Paster.prototype.init=function() {

 var paster = this;

 if (window.Clipboard) {
  //IE11, Chrome, Safari
  this.pasteEl.onpaste=function(e){
   paster.handlePaste(paster, e);
  };
 } else
 {
  //On Firefox use the contenteditable div hack
  this.canvas = document.createElement('canvas');
  this.pasteCatcher = document.createElement("div");
  this.pasteCatcher.setAttribute("id", "paste_ff");
  this.pasteCatcher.setAttribute("contenteditable", "");
  this.pasteCatcher.style.cssText = 'opacity:0;position:fixed;top:0px;left:0px;';
  this.pasteCatcher.style.marginLeft = "-20px";
  document.body.appendChild(this.pasteCatcher);


  this.pasteEl.onblur=function(e) {
   paster.pasteCatcher.focus();
  };

  this.pasteCatcher.onpaste=function(e) {

   paster.findImageEl(paster);
  };
 }
};

 
Paster.prototype.dataURItoBlob=function(dataURI, callback) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this

 console.log(dataURI);

 var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
 var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]

// write the bytes of the string to an ArrayBuffer
 var ab = new ArrayBuffer(byteString.length);
 var ia = new Uint8Array(ab);
 for (var i = 0; i < byteString.length; i++) {
  ia[i] = byteString.charCodeAt(i);
 }

// write the ArrayBuffer to a blob, and you're done

 return new Blob([ia], {type: mimeString});
};

Paster.prototype.findImageEl = function(paster) {

 if (paster.pasteCatcher.children.length > 0) {

  var dataURI = paster.pasteCatcher.firstElementChild.src;
  if (dataURI) {
   if (dataURI.indexOf('base64') === -1) {
    alert("Sorry, with Firefox you can only paste local screenshots and files. Use Chrome or IE11 if you need paster feature.");
    return;
   }

   var file = paster.dataURItoBlob(dataURI);
   paster.uploadFile(paster, file);
  }

  paster.pasteCatcher.innerHTML = '';

 } else
 {
  setTimeout(function() {
   paster.findImageEl(paster);
  }, 100);
 }
};

Paster.prototype.processing = false; //some wierd chrome bug makes the paste event fire twice when using javascript prompt for the filename

Paster.prototype.handlePaste = function(paster, e) {

 //don't do this twice
 if (paster.processing) {
  return;
 }

 //loop through all clipBoardData items and upload it if it's a file.
 for (var i = 0; i < e.clipboardData.items.length; i++) {
  var item = e.clipboardData.items[i];
  if (item.kind === "file") {

   paster.processing = true;
   e.preventDefault();
   paster.uploadFile(paster, item.getAsFile());
  }
 }
};


Paster.prototype.uploadFile = function(paster, file) {

 var xhr = new XMLHttpRequest();

 //progress logging
 xhr.upload.onprogress = function(e) {
  var percentComplete = (e.loaded / e.total) * 100;

  console.log(percentComplete);
 };

 //called when finished
 xhr.onload = function() {
  if (xhr.status === 200) {
   alert("Sucess! Upload completed. PHP response will be put in the textarea.");
  } else {
   alert("Error! Upload failed");
  }

  paster.processing = false;
 };

 //error handling
 xhr.onerror = function() {
  alert("Error! Upload failed. Can not connect to server.");
 };

 //trigger a callback when it's successful
 xhr.onreadystatechange = function()
 {
  if (xhr.readyState === 4 && xhr.status === 200)
  {
   if (paster.callback) {
    paster.callback.call(paster.scope || paster, paster, xhr);
   }
  }
 };


 //prompt for the filename
 var filename = prompt("Please enter the file name", "Pasted image");

 if (filename) {

  //upload the file
  xhr.open("POST", "<?php echo $_SERVER['PHP_SELF']; ?>", {
   filename: filename,
   filetype: file.type
  });
  
  //send it as multipart/form-data
  var formData = new FormData();
  formData.append("pastedFile", file);
  xhr.send(formData);
 }
};
</script>
</head>
<body>
 
 <textarea id="text" style="width:500px;height:500px;">
 
On chrome paste a screenshot in here and it will be uploaded.

On firefox first click outside the text area and paste the screenshot with CTRL+V

 </textarea>
 
 
 <script>
 
 //Create the paster object and connect it to the textarea.
 var paster = new Paster({
  pasteEl:document.getElementById("text"),
  callback:function(paster, xhr){
   
   //put the response in the textarea
   paster.pasteEl.value=xhr.responseText;
  }
 });
 
 //focus the textarea
 paster.pasteEl.focus();
 
 </script>
 
</body>
</html>

Thursday, March 6, 2014

Synchronize your Android phone with Group-Office without using Microsoft ActiveSync

Synchronizing Group-Office Professional using Microsoft ActiveSync is the easiest to setup and works great. But some might need an alternative. For those who don't know Group-Office yet here's a screenshot of the calendar:





I myself used to own a Samsung Galaxy S2 and I always happily synchronized this with Group-Office using Microsoft ActiveSync. I recently got myself a new LG Nexus 5 and I immediately setup ActiveSync again. But I immediately missed two important features:
  1. No tasks synchronization anymore! The stock Android OS doesn't have a tasks app. 
  2. I couldn't search for older mail anymore. The Samsung Galaxy had a very nice feature to search for mail on the server. So even when you only sync two weeks of mail, you could still find an older message by keyword. 
So I started looking for alternatives:

E-mail with IMAP

The e-mail solution was easy. I just setup an IMAP account that works out of the box with Android and Group-Office. Here's a screenshot of the Android e-mail app:



Contacts and Calendar sync

I found “Caldav Sync Free Beta” and “CardDav Sync Free beta” in the Google play store that were both easy to setup and work great. They are open source and free of charge as well!

The apps are implemented as a sync adapter so they seamlessly integrate with the standard android contacts and calendar application. The screenshot below just shows the standard Android calendar because that's all you will see and that's just the way it should be!


Tasks

Synchronizing tasks with Group-Office proved to be more difficult. I tried “CalDAV Tasksync beta free” and it worked, but I think it's in an early stage of development because the interface is not very good yet. But it looks promising for the future.

I also tried some other tasks apps that didn't work at all. I then decided to remove “Caldav Sync Free Beta” and replace it with “CalDAV-Sync” because it also has a nice tasks app with it. This app costs € 2,59 but it's worth the money! It also integrates nicely with the android calendar app and the tasks application looks and works great too!

Here's a screenshot of the Tasks app that synchronizes with Group-Office:



Ending

In lots of cases synchronizing Group-Office with Microsoft ActiveSync will be easier as you only have to setup one account for all. But the CalDAV alternatives work great too. It's just that you have to enter your new password four times when you setup a new one ;)

But as I did my research on these apps I came across great open source projects that might be the base of a combined tasks, calendar and contacts sync app for Group-Office!

Thursday, February 20, 2014

Creating your own project and time tracking reports

Introduction

With the new projects version 2 module we have created a very powerful and flexible reporting system. We supply open source reports written in PHP so anybody with PHP knowledge can create their own reports.

The reports that come with Group-Office are located in “modules/projects2/report”. There are already reports about time tracking of employees, project information and a general overview of all projects.

All these reports extend a simple abstract class that holds some basic functionality.



Creating the PHP code

It's best to keep your own reports in your data folder (defined in $config['file_storage_path']='/home/groupoffice'). The projects module automatically detects reports in “/home/groupoffice/php/projects2/report”.

So if you would create a report called “MyReport” then you must create the file “MyReport.php” there. Remember that everything is case sensitive!

In MyReport.php define the class name like this:



The start “GOFS” in the class name will instruct the Group-Office class loader to look in the home folder for this report.

In the abstract class there are 3 methods that will control when this report template will be available:

  1. supportsSelectedProject
  2. supportsBatchReport
  3. supportedProjectTypes

It can show up when you run a batch report from the main toolbar in the module or when a project container or project was selected through the “New” → “Report” menu option.

In the following example we create an Excel report using the PHPExcel library.



<?php
/**
 *
 * The MIT License (MIT)
 *
 * Copyright (c) 2014 Intermesh BV 
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 *
 */

/**
 * Simple XLSX report for project information
 *
 */
class GO_Projects2_Report_MyReport extends GO_Projects2_Report_Abstract {

 /**
  * Return filename extension.
  * @return string
  */
 public function fileExtension() {
  return 'xlsx';
 }

 /**
  * Return the name of the report used in the drop down
  * 
  * @return string
  */
 public function name() {
  return 'My Excel report';
 }

 /**
  * With this enabled it will only show up if this report is created from the
  * "New" menu in a project.
  * 
  * @return boolean defaults to false
  */
 public function supportsSelectedProject() {
  return true;
 }

 /**
  * With this enables it shows up when you create a report from the main toolbar
  * in the projects module.
  * 
  * @return boolean
  */
 public function supportsBatchReport() {
  return false;
 }

 /**
  * Indicates which project types are accepted for this report.
  * @return boolean Array containing for example: GO_Projects2_Model_Template::PROJECT_TYPE_PROJECT or GO_Projects2_Model_Template::PROJECT_TYPE_CONTAINER
  */
 public function supportedProjectTypes() {
  return array(GO_Projects2_Model_Template::PROJECT_TYPE_PROJECT);
 }

 /**
  * Indicate whether this report supports a start and end date
  * 
  * @return boolean defaults to false
  */
 public function supportsDateRange() {
  return false;
 }

 /**
  * Indicate whether this report supports a start and end date
  * 
  * @return boolean defaults to false
  */
 public function supportsStatusFilter() {
  return false;
 }

 /**
  *
  * @var GO_Base_Util_Excel 
  */
 private $_xls;

 /**
  * Start rendering the report.
  * 
  * @param boolean $return to return the file data as string.
  */
 public function render($return = false) {


  $this->project->id;

  //Create an Excel object. See http://www.codeplex.com/PHPExcel) for more 
  //information
  $this->_xls = new GO_Base_Util_Excel();

  $this->_xls->getProperties()->setCreator(GO::user()->name);
  $this->_xls->getProperties()->setLastModifiedBy(GO::user()->name);

  $this->_xls->getProperties()->setTitle($this->name());
  $this->_xls->getProperties()->setSubject($this->project->path);

  $this->_xls->setActiveSheetIndex(0);
  $this->_xls->getActiveSheet()->setTitle("Project information");


  $this->_xls->setDefaultStyle('Arial', 12);
  $this->_xls->setDefaultWidth(15);

  $this->_xls->setCellValue('A1', GO::t('strName'));
  $this->_xls->setCellValue('B1', $this->project->path);


  $this->_xls->setCellValue('A2', GO::t('status', 'projects2'));
  $this->_xls->setCellValue('B2', $this->project->status->name);

  $this->_xls->setCellValue('A3', GO::t('startTime', 'projects2'));
  $this->_xls->setCellValue('B3', $this->project->getAttribute('start_time', 'formatted'));


  $this->_xls->setCellValue('A4', GO::t('projectDue_time', 'projects2'));
  $this->_xls->setCellValue('B4', $this->project->getAttribute('due_time', 'formatted'));

  $this->_xls->setCellValue('A5', GO::t('lag', 'projects2'));
  $this->_xls->setCellValue('B5', GO_Base_Util_Date::minutesToTimeString($this->project->getLag()));

  $this->_xls->setCellValue('A6', GO::t('strDescription'));
  $this->_xls->setCellValue('B6', $this->project->description);
  $this->_xls->getActiveSheet()->getStyle("B6:B6")->getAlignment()->setWrapText(true);


  $file = GO_Base_Fs_File::tempFile($this->filename, $this->fileExtension());


  $this->_xls->save($file->path());


  if ($return) {
   //we must return it as string   
   $content = $file->getContents();
   $file->delete();
   return $content;
  } else {

   //we output it to the browser
   GO_Base_Util_Http::outputDownloadHeaders($file);
   $file->output();
  }
 }

}


Ending

I hope this simple example gets you started with project reports. The existing PDF reports are also a great resource to help you get going.