RavenDB on Linux - Source Code

I've created a new github repository to temporarily host the updated code for RavenDB to enable execution under Linux. I say temporary because a few things could happen.

Worst case scenario, and I don't believe this would happen, but there is a slight possibility, that I might be asked to shut down the repository because I unknowingly violated some terms of use. For an open source project, this is a highly unlikely scenario.

The normal case scenario would be that I get no notice from the creators of RavenDB in which case this code would continue to exist under its own repository.

The bese case scenario, and I would really like this to be the case, would be that my changes in some shape be accommodated upstream in RavenDB making RavenDB a cross platform tool.

I did not investigate why the OutputStream.Flush() command was causing an exception. At the same time, this is really my first attempt at MEF and .Net 4.0 and I don't know why the exports were not automagically loaded, in resolution to which, I had to manually load them using reflections. A better fix would be to identify and resolve these issues.

I am glad, however, that I was able to fulfill a personal quest of learning about RavenDB, and in the process, making it run under Linux. This opens up the possibility of making RavenDB a serious contender against MongoDB on the non-Windows platforms.

RavenDB along with my source code changes are available at https://github.com/jimmy00784/RavenDB-for-Linux https://github.com/jimmy00784/ravendb.

Note: Source code url updated.


This article is part of the series NoSQL - RavenDB on Linux. The series contains the following articles:
NoSQL - RavenDB on Linux
Open Source Shines - RavenDB on Linux
RavenDB on Linux - Source Code
RavenDB on Linux - Update

Open Source Shines - RavenDB on Linux

In the previous article, I wrote about RavenDB - a .Net NoSQL Document Oriented Database that supports Transactions - and why I had decided to spend time on it to get it to run under Linux. Now RavenDB was created for a Windows only audience and the creator's website doesn't not provide any guidelines about how to implement it under Linux.

After numerous attempts and with some modifications to the source code, I was able to get it to run under Linux. Some features that are not available under Linux as they are under Windows such as the web based UI - Raven Studio . However, dynamic index functionality also did not seem to work under Linux for some reason.

In my initial review had failed to notice something. When the server application starts at command line, you have a few options such as garbage collect, clear screen, and reset. When I had fired the test application that I had created, I noticed and then did not pay any attention to the fact that the server was not sending responses to all of the calls that were being sent by the client. When this happened, the client froze until either I issued a reset command to the server command line, or the connection timed out eventually. Since reset seemed to get the ball rolling for the moment, I was not too much concerned at that time.

The couple days ago, I decided to spend time to finish the task that I had started - to tweak RavenDB's code enough to get the database to fully work under Linux and then to use it in a project. My many efforts to identify the root cause of the issue were not at all fruitful. My first instinct was to review the server log files. There was nothing useful to be found there.

I thought that may be the document Id generator was causing the hold up so I provided manually generated Ids to my document objects. When that didn't solve the issue I started placing breakpoints all over the client library code and debugged the application one line at a time, stepping into every function call possible. I came across a piece of code that seemed like a promising lead.

It was in the Raven.Client.Lightweight project in the Connections/HttpJsonRequest.cs file at line 304.

var text = reader.ReadToEnd();

When control reached that statement, the execution froze until I issued the reset command at the server command line. It meant only one thing - there had to be a corresponding response writing activity on the server. I reviewed the code couple lines above to get some clues. I evaluated response object on line 299 for its ResponseUri property.

ResponseHeaders = response.Headers;

The ResponseUri property was set to "http://localhost:8080/bulk_docs".

At the same time the server had the following output on the terminal:

That was my clue. I needed to locate a responder that handled the "bulk_docs" url.

Raven.Database project has a responder that handles "bulk_docs" under Server/Responders/DocumentBatch.cs. On setting breakpoint in the file on line 38 and following the control line by line lead me to Server/HttpServer.cs on line 318 which always ended in exception with a not so meaningful error message and once this method was executed the control stop.

I debugged further into the FinalizeRequestProcessing which led me to line 351.

ctx.FinalizeResonse();

This took me to Server/Abstractions/HttpListenerContextAdapter.cs line 82. The control disappeared once I tried to step further than like 86.

ResponseInternal.OutputStream.Flush();

I put a breakpoint in the catch block and sure enough there was an exception - and I/O exception of some sort - which was not handled. Couple lines below was the smoking gun. Line 88 never executed in event of an exception and that would be what was causing the server to hang on to request.

ctx.Response.Close();

I made a slight tweak to this file and moved the two lines into the finally block.

I recompiled the server and there it was. RavenDB was not longer freezing on individual requests and dynamic indices were working as designed. RavenDB on Linux is now ready to be used in application development. Since RavenDB was created as an Open Source application, it was possible to review the code and troubleshoot this issue. The issue seems to not affect Windows as RavenDB is being used by real world businesses.

Next thing to do would be to create a non WPF replacement for Raven Studio for use in Linux.


This article is part of the series NoSQL - RavenDB on Linux. The series contains the following articles:
NoSQL - RavenDB on Linux
Open Source Shines - RavenDB on Linux
RavenDB on Linux - Source Code
RavenDB on Linux - Update

NoSQL - RavenDB on Linux

For a while now I've been on a quest to find a NoSQL database that met the following criteria:

1. Document Oriented Database
2. Supports Transactions across multiple Documents
3. .Net/Mono compatible drivers
4. Runs under Linux

And the journey has been anything but easy.

Ever since I first read about Document Oriented NoSQL Databases, I've been fasinated by them. I started looking into CouchDB [apache.org] at first, but MongoDB [mongodb.org] soon became my favorite.

MongoDB is a Document Oriented Database that is written in C++ with performance in mind. MongoDB allowed my to write applications using my favorite languages and ran on many Operating Systems. After doing some research, I also found a port of MongoDB for ARM. It now runs on a Debian server that I have setup on PogoPlug which runs on an arm chipset. The JSON style BSON representation of data was simple to follow and the ability store any object in MongoDB without having to implement special interfaces or inheirit from special classes made it that much more appealing to me. The .Net drivers were readily available on their website along with plenty of documentation on how to use them.

MongoDB met three out of the three criteria I had above. The was one that it didn't satisfy: MongoDB doesn't yet support Transactions [mongodb.org] and there is no word out yet that would promise availability of transactions any time soon.

I continued my research and came across RavenDB [ravendb.net]. RavenDB is a Document Oriented Database that is written completely in .Net. It is also Mono compatible. Unlike MongoDB, RavenDB does support transactions across multiple documents, however, it only runs under Windows. Though evn RavenDB doesn't meet all the four criteria, since it was written completely in .Net, that gives me something to tinker around and see if it could be made to run under Linux.

My first attempt was a few months ago when I first read about RavenDB. I downloaded the code from their GitHub git repository and fired up MonoDevelop [monodevelop.com]. The issues became immediately apparent: heavy reliance on Silverlight. But even then, there was .Net 4.0 code that my mono compiler could not make sense of. After spending few minutes over it, I gave up on it.

Few days ago, with the update of Ubuntu, I got the near latest version of Mono installed on my laptop. I decided to give RavenDB another try.

From my konsole, I issue the command to get the latest code for RavenDB and created a new branch - linux.

git clone git://github.com/ravendb/ravendb.git

cd ravendb

git branch linux

git checkout linux

As soon as I fired up MonoDevelop and loaded the solution, I was greeted with this error message:

Also, the projects Raven.Backup and Raven.Smuggler were not set to build under my current configuration. I noticed that upon Right Click -> Options -> Build -> Configuration on the Raven.Backup project, the only configurations that were available were for x86. I was able to select Debug, click Copy button and create another configuration for All CPU. I repeated this for Release and then the same steps for the other project as well.

Then from the Right Click -> Options -> Build -> Configurations -> Configuration Mapping tab, I selected the correct configuration for the two projects. now the projects were no longer marked as "unable to build under current configuration." Now, on to the issue with the three projects failing to load: Raven.Client.Silverlight, Raven.Studio, Raven.Tests.Silverlight.

While anything Silverlight under Linux (using Moonlight) was not a promising prospect to begin with, I still wanted to give it a shot before beginning the removal of non-compilable projects. I opened the project files in plain text format and searched for the GUIDs from the screen print above. On finding them, I commented them and attempted to reload the projects. The solution was now ready for the first compile.

First attempt:

95 build errors. I also noticed that at least a few of those errors were really warnings. This meant one thing - some or all the projects were setup to treat warnings as errors. That was an easy fix. On all the projects, I unchecked the "Treat warnings as errors" box from Right click -> Options -> Build -> Compiler screen. Time for the second compile attempt.

Second attempt:

100 build errors. One of the first errors was in the Raven.Client.Debug project on classes from the Microsoft.VisualStudio.DebuggerVisualizers namespace. Dependency on Visual Studio would be a problem under Linux. I decided to not dwell too much on this error and chose to remove this project all together from the solution. All the dependencies on Raven.Client.Debug would also have to be removed.

Third attempt:

95 build errors. This time it was the NLog namespace under the Raven.Tryouts project. Reviewing the error revealed that NLog dll was not compatible with the current Mono runtime. Fortunately, NLog had a Mono compatible binary available for download on Codeplex. I downloaded it and replaced the reference in the project.

Forth attempt:

94 build errors. Upon closer inspection, I observed that Raven.Web relied on System.Web.Entity which is not available under Mono. Also, Raven.Client.Silverlight System.Windows and System.Windows.Browser which are also not included with Mono. I decided to remove these two project from compilation along with Raven.Tests.Silverlight project. Raven.Studio project was the next one to go since it was a WPF application - lots of XAML files - and WPF is not fully implemented under Mono.

Fifth attempt:

0 build errors. Are we there yet? Let's give it a try. I ran the Raven.Server project. Bummer! Runtime exception. DllNotFoundException. Turns out that Raven.Storage.Esent project implements Microsoft's ISAM Esent storage which is proprietary and requires Windows in order to run. Since we also had Raven.Storage.Managed project, I decided to modify the configuration of the application to use the managed storage library Munin instead of UnManaged Esent. I modified the App.Config under Raven.Server and changed the value for "Raven/StorageEngine" to "Munin" from "Esent".

Sixth attempt:

0 build errors (expected). Yet another runtime error however. This time it was a MissingManifestResourceException with a very vague description and a long stack trace. However, right before the stack trace took me into obscurity, there was a hint - ravendb/Raven.Database/Server/HttpServer.cs:110. The SatisfyImportsOnce call was failing after some execution. After doing some reading into it, I found out that RavenDB was build on MEF which is a feature of .Net 4.0 and is supposed to provide easy plugin/extension functionality to .Net programs. That line was supposed to initialize the extensions once. Somewhere in the code below there would be a code block that would expect non null values. I commented that line and proceeded.

Seventh attempt:

0 build errors. As expected, NullReferenceException on Raven.Database/Server/HttpServer.cs:112. RequestResponders was supposed to be non-null. I did some more research to understand how MEF was supposed to implement the extensibility. I had to find classes that were inheirited from AbstractRequestResponder. May be manually loading objects into RequestResponders would help. And indeed I found plenty of classes that were inheirited from AbstractRequestResponder and RequestResponder classes. I added the following lines of code in HttpServer.cs

Eight attempt:

0 build errors. Another exception, this time NullReferenceException at Server/HttpServer.cs:193. This seemed similar to the previous issue but with a different object - ConfigureHttpListeners. Time to repeat the exersice with different base class - interface this time.

Ninth attempt:

0 build errors. I started seeing some output repeating and they didn't look like error messages:

Available commands: cls, reset, gc, q

Could not understand:

I decided to run the compiled application from konsole to see how it would behave there.

Raven is ready to process requests. Build 13, Version 1.0.0.0 / abcdef0

Server started in 847 ms

Data directory: /home/karim/Projects/RavenDB/ravendb/Raven.Server/bin/Debug/Data

HostName: <any> Port: 8080, Storage: Munin

Server Url: http://karim-laptop:8080/

Available commands: cls, reset, gc, q

Time to run some tests. I created a small console .Net application and decided to fire it up. It worked! Some issues did surface, but nonetheless, it worked. I was able to store and retrieve simple documents to the database. Since I had removed some of the critical projects such as Raven.Web and Raven.Studio, the web interface was gone with them. Dynamic indices did not work either.

Now that I have a semi functional database, I'll put some time in and try to create a simple Raven Studio replacement as well as work the other kinks out.

Overall, it helps that the application was written completely in .Net without any dependencies on the OS that could not be achieved via framework level abstraction. I am certain that RavenDB could be made to work under Linux at similar level of confidence as it does under Windows.

This article is part of the series NoSQL - RavenDB on Linux. The series contains the following articles:
NoSQL - RavenDB on Linux
Open Source Shines - RavenDB on Linux
RavenDB on Linux - Source Code
RavenDB on Linux - Update

CallCentric - VoIP Service Provider

CallCentric
CallCentric offers variety of VoIP products for personal, residential, as well as office use. If you need to switch your traditional phone lines to SIP save money on phone bills, or if you just need additional phone numbers besides your traditional phone lines, CallCentric has products offerings that rival those from their competitors.
CallCentric numbers can be very easily:

1. configured on the IP Phones on your network
2. forwarded to traditional phones
3. routed to your internal PBX, like FreeSWITCH

Call forwarding from CallCentric to SIP address is free, so are calls from one CallCentric number to another. Local numbers are available. You can even transfer your existing number to CallCentric. Signup for a free account here. Additional paid features can be added at any time.

Filesystem Encryption under Linux - EncFS

Customary disclaimer: The author takes absolutely no responsibility for any corruption or loss of data, especially those that might result out of application error, user error, or user negligence. Please also check your local laws pertaining to sharing encrypted data as it might be illegal in some countries. The author is not a lawyer and the contents of the article should not be perceived as legal advice under any circumstance. Please use at your own discretion.

In the previous article, I gave a quick introduction about FUSE and how ClamFS could be used to secure filesystem against malware. Another area where FUSE shines is in the field of encrypted filesystems. There are quite a few options available when it comes to setting up encrypted storage for safe keeping of important content. In this article, I'll discuss one such encrypted storage option that is built using FUSE functionality: EncFS.

EncFS is a simple to setup virtual filesystem that provides portable encrypted storage for your data. It is a pass-through filesystem, which means that while the files themselves exist on the filesystem encrypted, with EncFS loaded, they appear decrypted. Unlike many other encrypted filesystems which require you to dedicate entire partitions for encrypted storage, EncFS exists on top of existing filesystems. You don't need to pre-allocate fixed amounts of space for it, hence, there is no wastage of storage.

You can install EncFS easily from your software package manager. On Debian based distributions it is as simple as typing the following on the terminal window and providing the sudo user's password when prompted (exclude the $ sign):

$ sudo apt-get install encfs

Once it is installed, you can create your encrypted storage and a decrypted mount point with the following command:

$ encfs path-to-encrypted-storage path-to-decrypted-mount-point

Example:

$ encfs /tmp/encrypted /tmp/decrypted

You will be prompted if the folders should be created if they don't already exist. You will also be prompted to chose the level of encryption. Your choices are simple, paranoid, and expert.

Please choose from one of the following options:
enter "x" for expert configuration mode,
enter "p" for pre-configured paranoia mode,
anything else, or an empty line will select standard mode.

Once you have provided your choice, you will be prompted to enter and confirm a password to enable access to the encrypted storage. Do not forget this password or you will lose your encrypted data. You can now start using your encrypted storage. Anything that is saved under the decrypted mount point is actually encrypted and saved under the encrypted storage. The filenames as well as the content of the files will be encrypted. Do not save data directly under encrypted storage as data written directly there does not get encrypted.

Once you are done securing your data, you can unmount the decrypted mount point by using the fusermount command as follows:

$ fusermount -u path-to-decrypted-mount-point

To remount the encrypted storage, just reissue the encfs command and provide your password:

$ encfs path-to-encrypted-storage path-to-decrypted-mount-point


Conclusion

EncFS' simplicity to use makes it an ideal tool to help provide ad-hoc encryption capability on existing filesystems without having to spend any money on additional hardware. EncFS is also very flexible and portable, and allows you to store encrypted data along side with unencrypted data. It is however necessary to point out that, like any other encryption scenario, it is important to maintain secure unencrypted backup copies of critical data for emergencies like filesystem and hardware failures etc.

You may also like the series - ClamAV - Antivirus for Linux:

ClamAV - Antivirus for Linux
ClamFS - Antivirus Filesystem for Linux

ClamFS - Antivirus Filesystem for Linux

Disclaimer: Flying through Linux and OpenSource. You might experience Freedom along with plenty of awesomeness.

Linux as never failed to amaze me with its simplicity and its feature richness. That was one of the few reasons that convinced me to switch to Linux on my computers as the primary Operating System. One such feature is the ability to load virtual filesystems in userspace using the FUSE module. When enabled, the custom FUSE module would allow you to load custom data stores that could be traditional and non-traditional file systems or custom programs that allow filesystem like interaction. You could then interact with these custom userspace filesystems as though they were like any other filesystem. Depending on the fuse module, you could browse the folders and create, modify, and delete data as though they were regular files on your computer.

Two of the many real world applications of FUSE are in the following areas:

1. Antivirus filesystems - Creates virtual filesystems that trigger antivirus scanning whenever files within the filesystems are written to or read from.


2. Encrypted filesystems - Creates virtual filesystems that automatically encrypt files on write operations and decrypt them on reads.

In this article, I'll introduce ClamFS, which lets you create a virtual antivirus filesystem on top of your existing filesystem.

ClamFS triggers automatic scanning of files using ClamAV whenever I/O is performed on them. The best candidate for ClamFS would be the default downloads folder where you Internet browser saves those files that are downloaded from the Internet. You would ideally start with an empty Downloads folder, however, ClamFS also allows you to secure folders that already have contents in them.

ClamFS could be easily installed from your Linux distributions software package manager. Just search for clamfs. For each virtual antivirus filesystem that you'd like to create, you will need an XML configuration file. The configuration file will let you define the following:

1. What folder to secure, where to mount the virtual filesystem
2. What maximum file size to scan - to increase performance
3. Whitelist extensions - files with these extensions will never be scanned
4. Blacklist extensions - files with these extensions will always be scanned regardless of maximum file size parameter
5. Logging method - standard out, syslog, file, or email

In the event that you attempt to download or write an infected file to the ClamFS virtual filesystem, an error entry will be logged and the file will not be successfully written to your filesystem. If an infected file already exists on the ClamFS virtual filesystem, you will not be allowed to read from it.

Here is an example configuration: clamfs.xml

The command to enable ClamFS protected virtual filesystem is:

$ sudo clamfs configuration.xml

You can find more information about ClamFS and on the configuration XML on their website.

This article is part of the series ClamAV - Antivirus for Linux:

ClamAV - Antivirus for Linux

ClamFS - Antivirus Filesystem for Linux

You may also like:

Filesystem Encryption under Linux - EncFS

ClamAV - Antivirus for Linux

Disclaimer: Flying through Linux and OpenSource. You might experience Freedom along with plenty of awesomeness.

In today's digital world, where access to electronic content is so convenient that it is almost taken for granted, there is a constant threat of malware infection.

While Linux computers a much less likely to be infected by a malware transmitted via sharing files via removable media or over the Internet, it is certainly not impossible. And while you yourself may not be too much concerned about getting your computer infected because of an infected file, you may unknowingly put the non-Linux users with whom you may share files at risk. These could be your family, friends, and colleagues, etc.

There are a few steps that one can take to prevent and minimize getting and spreading such infections and the most common one happens to be installing an antivirus software. Many antivirus software vendors today offer some paid as well as free versions for Linux computers. When I was a Window user, one of the first things I did after (re)installing the operating system on my computer was to install an antivirus on it. I hated using antivirus, especially the real-time scan features, since they slowed my computer, negatively affecting the overall experience. Ever since I made the switch to Linux few years ago I've not had that problem and I've gotten used to using my computers without any antivirus software installed. I changed my mind few weeks ago when I realized that malware were no longer restricted to the standard executable files. Malware authors are now exploiting vulnerabilities in popular software like Adobe PDF Reader and Flash Player etc. and packaging malware in pdf, swf, and other files. While Linux users should still be relatively safe from such infections, non-Linux users, especially Windows users are not. Therefore, in order to stop the malware from spreading out from your computer, it is essential that they be detected there first.

ClamAV is a popular open source antivirus that has been around for many years. Its website describes it as follows.

ClamAV is an open source (GPL) antivirus engine designed for detecting Trojans, viruses, malware and other malicious threats. It is the de facto standard for mail gateway scanning. It provides a high performance mutli-threaded scanning daemon, command line utilities for on demand file scanning, and an intelligent tool for automatic signature updates.

It is available on many popular Linux distributions. On Ubuntu, it can be installed using the Software Center or any Package Manager. Just search for clamav. You will also need to install the virus definitions updater for ClamAV which is known as freshclam. Keep in mind that ClamAV does not install a graphical user interface and once you've installed clamav and freshclam, they would be only usable via the command line shell. There are however a few GUI tools available that can be used with clamav and one such tool is clam tk which could also be installed from the software manager. Once installed, clamtk will allow you to scan files and directories using clamav. It will also allow you to configure clamav using its Advanced -> Preferences menu entry.

  

If you are wondering what that Last infected file was, I used a test virus file, EICAR.COM, that is readily available over the Internet and can be used to test if the antivirus is in fact scanning and detecting infected files.

While you could use clamav from both command line as well as using the ClamTK gui, most modern graphical Linux distributions also allow you to add a menu entry on the right click context menu to scan a file or a folder using ClamAV. Here is a screen shot from my laptop which runs Kubuntu 11.10 showing the context menu entry.

While no antivirus software will make your computer 100% safe and secure, they will certainly help. ClamAV is the antivirus of choice by most Linux server administrators and computer users. At the end of the day however it boils down to how responsible and cautious you yourself are. I hope you found this article helpful.

This article is part of the series ClamAV - Antivirus for Linux:
ClamAV - Antivirus for Linux
ClamFS - Antivirus Filesystem for Linux

You may also like:
Filesystem Encryption under Linux - EncFS

FreeSWITCH Implementation ideas

Disclaimer: Flying through Linux and OpenSource. You might experience Freedom along with plenty of awesomeness.

So you have a SIP Server/PBX setup with FreeSWITCH. What all can you do with it?

Use it in your Small Office/Home Office:

1. Use it for interoffice voice communication.
2. Use it for internal voice communications between satellite office locations.
3. Setup call routing for all incoming calls based on the time of day and availability of personnel.
4. Create an IVR menu for your customers/clients to assist with call routing and to provide valuable information over the phone, such as hours of operation, directions, etc.
5. Create an IVR menu for your customers/clients to assist them with account inquiry - some coding needed to interface with your backend datastores.
6. Auto-announce important messages or season's greetings to the caller before routing calls.
7. Setup automated calls to your customers/clients reminding them about useful information or deadlines.

Use it in your Home:

1. Use it as a Skype replacement for your immediate and extended families.
2. Route incoming calls intelligently directly to the intended recipient's SIP phone device.
3. Make outbound calls from any computer or SIP/soft phone on your network.
4. Block unsolicited incoming calls from telemarketers.
5. Block unsolicited incoming calls from annoying callers.
6. Play "This number is no longer in service" when annoying callers call.
7. Screen incoming calls having callers announce their names before sending the call through, giving you the opportunity to reject calls.
8. Setup reminders having the "home" call you reminding you about the milk.
9. Prank your friends by using pretend voice greetings when they call.
10. Replace your fixed home phone units with unused smartphones over WiFi.
11. Make calls using your home phone even when away (*see how I set this feature on my home network below)
12. Use you favorite VoIP home phone service's features from anywhere (*see how I set this feature on my home network below)

Fun project ideas that could use FreeSWITCH - some coding required:

1. Interface with your Home Automation System and turn lights on and off by calling your home.
2. Have your home phone read out your emails or rss feeds when you call.
3. Have your home phone take dictation and voice notes and store them on your home network.
4. Setup a dial-in radio/jukebox.
5. Have your home phone give you information about local weather, news headlines, your server's overall health, software application's progress, etc.

Here is how I use it in my home network:


We have a Vonage home phone serivce with a flat monthly rate international calling plan. However, it is a fixed home phone line. Now with the FreeSWITCH + SPA 3102 setup, I can make international phone calls from my Android cell phone over 3G and WiFi where ever available.
For all the non-smart phones in our household, I found a free service called IPKall  that gives you a regular phone number (Washington state numbers only) that can be forwarded to any sip phone. I've configured a new extension on the public profile and forwarded the IPKall phone number to go to this extension. The following happens when someone calls this IPKall number:

1. If the incoming caller id is identified as one of our cell phones, then the caller is prompted to enter the number. FreeSWITCH then initiates a outbound call using the home phone line and bridges the two calls.
2. If the incoming caller id is not identified as one of our cell phones, then the caller is sent to voice mail box.

This way my parents, who don't yet have an Android phone, can still make international calls through our Vonage home phone from their cell phones. At the same time I don't have to worry too much about someone finding out about this IPKall number and using it to make unauthorized international calls using our phone line.


This article is part of a series which can be tracked by the following articles:
PogoDebian + FreeSWITCH + Linksys 3102 VoIP Gateway = Pure Bliss - Part 1
Hello!!! We have a dial tone.
"Operator. How may I direct your call?"
FreeSWITCH Implementation ideas

"Operator. How may I direct your call?"

Disclaimer: Flying through Linux and OpenSource. You might experience Freedom along with plenty of awesomeness.

I am a curious being and I love the freedom that is offered by Linux. Because of this, you will occasionally find me spending hours on pet projects that have nothing to do with what I do for a living.

In past couple posts, I shared my desire and my efforts to set up my own PBX in my home network. I also shared the discussion I had with a colleague who had successfully setup an intercom system that he used to talk with his family abroad. After spending numerous hours on this feat over past few days, last night I finished my setup and I now have all the functionality that I needed from this setup.

I installed FreeSWITCH on a Debian server that I have on my home network. It had to be compiled from source since PogoPlug houses an ARM processor and Debian repositories didn't have a packages for FreeSWITCH for arm architecture. But the instructions were easily available and installation was a piece of cake. Like I had mentioned in the previous post, the installation came pre-configured with 20 extensions for internal use, an demo IVR menu, and plenty of other features.

I was now able to register soft phone softwares running on our Linux desktops and laptops and my Android phone over WiFi within our home network and test the calling from one soft phone extension to another.
I now needed a sip gateway router that would allow me to connect my home phone line to the FreeSWITCH setup. I found a great deal on a Cisco/Linksys SPA 3102 sip gateway router. SPA 3102 allowed me to connect my incoming phone line on one jack and my home phone unit on the other.
I now had a PBX setup which was accessible to be via the home network for internal calling. I was also connected to my home phone line, which allowed me to make outbound calls and receive incoming calls.
Now one thing to keep in mind that even though SPA 3102 has a jack for phone line (FXO) and a jack to connect a physical phone (FXS) they operate on two separate extensions. However, now that we are in SIP/PBX territory, calls can be bridged and transferred from one extension to another. I was able to configure FreeSWITCH and SPA 3102 so that the calls received on the PSTN line (FXS) would be forwarded to Line 1 (FXO) where my phone was attached. I also configured FreeSWITCH to send the calls originating on internal extensions - Line 1 on SPA 3102 also being one of them - that are not handled internally, to send them outbound via PSTN line. My basic phone setup was restored and I now had the capability to make outbound calls from any computer from within my home network. Think Skype Out without paying an addition cent.

I mentioned earlier that I was able to register and test soft phone on my Android phone to my home FreeSWITCH server. This was done over WiFi. I was able to open ports on my home firewall for FreeSWITCH so that I could connect from an outside network. And I was successfully able to test this by getting on a different WiFi network that had Internet access. The next thing naturally was to do a quick test over 3G we'd be in business. Yeah, right. This was not as quick as I had anticipated it to be. I was able to originate outbound calls and receive incoming calls successfully, but there was no audio. After spending hours researching it, I found the answer hidden in plain sight, mocking me all this time.

SIP Servers and phones use what is called a STUN service to help with port discovery when connecting to other servers and phones behind NAT/firewall. I had already read about it briefly and had configured my server to use stun.freeswitch.org as the STUN service url. From the reviewing the logs, I discovered that the url was not reliable and down most of the time. There are however a number of free STUN services available and I found a webpage that listed quiet a few of those. FreeSWITCH comes with a STUN server testing utility and I was able to find one of the STUN servers that was indeed online. I configured my FreeSWITCH server to use it, and now I had audio on my Android phone over 3G.

All this was a fun and exciting experience overall. Besides the time I put in it, it cost me $43 that I paid for the SPA 3102. What are the practical applications you'd ask. There are plenty, and I plan to dedicate the next article discussing those.

Progress on this topic can be following in the following articles:

PogoDebian + FreeSWITCH + Linksys 3102 VoIP Gateway = Pure Bliss - Part 1

Hello!!! We have a dial tone.

"Operator. How may I direct your call?"

FreeSWITCH Implementation ideas

Hello!!! We have a dial tone.

Disclaimer: Flying through Linux and OpenSource. You might experience Freedom along with plenty of awesomeness.

After doing some initial reading up on FreeSWITCH. The reviews were promising enough for me to proceed with the installation on the Debian server that I had earlier setup on PogoPlug.

The documentation on the freeswitch.org was plentiful with beginners' guides and quick start manuals readily available and easy to follow. I couldn't install FreeSWITCH from the official debian repositories since they didn't host installation for ARM architecture. However, I was able to simply download the source and compile it on the server without any issues using the instructions that came included with the source code.

Once installed, there was really nothing to configure as I found that the out of the box setup was pretty decent. It came with 20 internal extensions - 1000 through 1019, a conference extension - 888, and a demo IVR menu extension - 5000.

I changed the default password and tested VoIP on the new PBX using QuteCom on my laptop and 3CX on my Android phone over WiFi within the home network. Initial results were very promising. I configured my router to allow SIP traffic through the firewall, again after reading up about it in the documentation. I was able to successfully test connectivity and voice from outside the network as well.

Now I got curious and decided to buy a VoIP gateway hardware that would allow me to connect my phone line in (FXO) and also my home phone device (FXS) and route calls from internal SIP to PSTN and vice versa. Luckily I got a very good deal on Linksys/Cisco SPA3102. After it arrived in the mail, I started the tedious task of connecting and configuring it.

I realized that I had exhausted all four of my Ethernet ports on my home DSL router. I had an old four port Linksys broadband router sitting in the attic. I turned off WiFi on it and bridged it to the home DSL router using an Ethernet cable. Now I had room for two more devices, one of which was going to be this new SPA3201 gateway server.

I began configuring the gateway. First, I tested receiving my home phone calls on my computer. Then I configured making outbound calls using my computer. So far so good.

Calling out using the phone handset was a piece of cake, however, receiving inbound calls on the phone was something else. After spending yet another couple hours on it, I was able to fix the configuration and receive inbound calls as well. I was able to test the IVR menu as well as make calls using my computer to test the scenario.

I'll be experimenting with the PBX setup and the dial-plans in the coming weeks to further explore its power. I had a lot of fun setting it up. Now to come up with ideas to put it to real world use.

Progress on this topic can be following in the following articles:

PogoDebian + FreeSWITCH + Linksys 3102 VoIP Gateway = Pure Bliss - Part 1

Hello!!! We have a dial tone.

"Operator. How may I direct your call?"

FreeSWITCH Implementation ideas

PogoDebian + FreeSWITCH + Linksys 3102 VoIP Gateway = Pure Bliss - Part 1

It was few years ago - around the year 2000 - when I had first toyed with the idea. The modem driver installation CD came with free PBX software that was IVR capable. I was amazed at what I had in my presence. Even back then, I was well aware of the possibilities. I was later reinitiated to Linux and orientation to Free Software came much later. Few months ago I discovered Asterisk but was put away by the expensive Digium hardware that it would need to operate, while still unware of what all Asterisk could offer.

Few weeks ago, I was in Florida for work and after learning that one of my colleagues also used Linux for a pet-projects of his, we both got into discussing how we had implemented our Linux setups. I told him about my Kubuntu laptops and the Debian server I was running on PogoPlug (I call it PogoDebian) and he told me about his SIP gateway server that he setup for VoIP telephony for his family in Sri Lanka. He uses it to talk with his family in US and in Sri Lanka. Moreover, he had a SIP phone application on his Android phone that he could use to connect to his server and make calls via his gateway at home over WiFi, even over 3g. He also mentioned about a VoIP gateway server from Linksys/Cisco that allowed him to connect a regular PSTN phone line to the SIP server and route his land line phone calls to his cell phone and vice versa. Spiked your interest? Yeah! I had the same feeling.

I made up my mind about it. I wanted to setup PBX like functionality in my house. A couple IVR menus would be neat as well. Some intelligent schedule based call routing and we'd be in business.
When I got back from that trip, I started my research on SIP servers and VoIP gateway. I found FreeSWITCH and was impressed by what it had to offer. It was capable of everything I had on my mind. As for the VoIP Gateway, Linksys/Cisco 3102 was ideal. It provision for 1 PSTN line and 1 phone/fax device. One key requirement that I had was even with the 3102 now in between our phone line and the phone itself, the existing phone configuration should not be lost and our home phone connect to the PSTN line should not change in the way it behaved.

Progress on this topic can be following in the following articles:
PogoDebian + FreeSWITCH + Linksys 3102 VoIP Gateway = Pure Bliss - Part 1
Hello!!! We have a dial tone.
"Operator. How may I direct your call?"
FreeSWITCH Implementation ideas