Update 1 - Progress so far - IrrlichtDotNet - .Net library for irrlicht that works under Linux

For original article, please go to this article.

After I made the declaration of starting this project on irrlicht forums, roxaz from the forums expressed interest in the project and immediately forked it on github. We bounced around ideas and while I won't yet merge all of his modifications into my repo, I did like many of this ideas and implemented them after tweaking them to fit in with my code. The ideas that I implemented from his repo are:

1. folder reorganization
2. a python script that refreshes the CS project file with all the SWIG generated cs files
3. make.sh file to automate many initial steps

I also spent some time getting myself more familiar with SWIG typemaps and was able to map irr::core::stringc to System.String for almost all instances. These originally got translated to a SWIG generated generic wrapper which wasn't any useful unless you created a companion helper class in C++ to handle the translation both ways between irr::core::stringc and char *, which gets mapped correctly to System.String. Once all occurrences of irr::core::stringc are mapped to System.String, helper class won't be necessary any more. I will also attempt to deploy more typemaps to handle other such mappings in IrrlichtDotNet project.

Linux compatible .Net bindings for irrLicht using SWIG

Ever since I tried IrrLicht, an Open Source 3d Graphics library, I fell in love with it. It is a C++ library and my C++ isn't as good as I'd like it to be and I am more comfortable with C# under .Net.

Since IrrLicht is open source, I figured, someone might have written an .Net binding for it, and I was correct. A few such bindings do exist, although, they are truly only .Net bindings for IrrLicht for Windows platform only.
I also attempted to use MonoGame, which is known to work under Linux, but without any success.

A few days ago, while researching an unrelated subject, I stumbled across SWIG. SWIG stands for Simple Wrapper and Interface Generator. It is used when you want write wrappers to your C/C++ code for other languages. It also happens to support C#. So I decided to give it SWIG a swing and initial results were more than promising.

I decided to test it but exposing the Main method of my C++ test code written to try IrrLicht and called it from a .Net application. It worked!
I decided to take a shot at generating a wrapper for IrrLicht itself. So far, using the bindings that I was able to create with SWIG, I have been able to create simple 3d scenes in C# code.

I've created a project on GitHub to create a Linux compatible .Net binding for IrrLicht using SWIG. The repo is at https://github.com/jimmy00784/IrrlichtDotNet.

Initially, much of the focus will be on creating a Linux centric binding. However, since SWIG is cross platform, if the community is interested in the effort, it could become more cross platform than that.

Day 2 - Porting nopCommerce to Linux - built in data storage

For some odd reason, I was now able to debug from within MonoDevelop again and that made me happy. Now that the Installation page loaded correctly, I decided to test the SqlServerCe built in data storage for nopCommerce. I provided the email and password, unchecked the "Create sample data" checkbox, selected the "Use built-in data storage (SQL Server Compact)" option and hit Install. Failed.

Setup failed: The requested feature is not implemented.

I set my breakpoint on the InstallController.cs and began walking through the code line by line to see where it had failed. I landed on SqlCeConnectionFactory.cs file line 41.

Contract.Requires(!string.IsNullOrWhiteSpace(providerInvariantName));

I tested the value of the providerInvariantName parameter to see if it was indeed null or whitespace, but it wasn't. For now I decided to comment this line and see if I could proceed. Same error. Examining the stack trace I found that Database.cs in the EntityFramework project had another Contract clause.

Contract.Requires(value != null);

And again, value wasn't null. It seemed like Contract clause wasn't behaving as designed. I decided to comment this one out as well. I found another one in EntityFramework, and then another one. After commenting out quiet a few of these over the next few trial runs, I decided to comment them all using Find and Replace. After fixing some compiler errors EntityFramework compiled without errors. Time to commit changes.

In next run, the application complained that it couldn't find SqlServerCe library. I noticed that Nop.Data project had an entry for System.Data.SqlServerCe.Entity.dll which had dependency issues under Linux. It appeared as though I couldn't get too far with SqlServerCe as the backend driver. I decided to give MS SQL Server a try instead.

In first try I got the familiar Feature Not Implemented error.

Setup failed: The requested feature is not implemented.

However, an empty database did get created. I did a search and commented Contract.Requires from the EntityFramework project. I also set a breakpoint on the FailFast  method in the Environment.cs core file which gets invoked when Contract clause failed.

Setup failed: The provider did not return a ProviderManifest instance.

Examining the stack trace hinted to this location: EntityFramework/Core/Common/DbProviderServices.cs:208.

There was also an inner exception being raised.

Invalid URI: The format of the URI could not be determined: System.Data.Resources.SqlClient.SqlProviderServices.ProviderManifest.xml

Further exploring down the stack trace took me to EntityFramework.SqlServer/SqlProviderManifest.cs:78 which passed a baseURI parameter. For some reason Mono didn't like the value that was being passed. I used an overloaded method that did not require baseURI to see if it'd let me proceed.

Setup failed: The requested feature is not implemented.

This was being raised from the EntityFramework.SqlServer project. I did another search and comment on Contract.Requires this time on the EntityFramework.SqlServer project.

System.StackOverflowException: The requested operation caused a stack overflow.

After another round of placing breakpoints and tracing through the code, I found that the stack overflow was being caused by EntityFramework/Core/Common/Utils/Memorizer.cs:71 when attempting to evaluate result.GetValue(). I surrounded it with a try catch and returned the default value of the generic TResult in case of an exception to see if it'd let me proceed.

Setup failed: An unexpected exception was thrown during validation of 'Name' when invoking System.ComponentModel.DataAnnotations.MaxLengthAttribute.IsValid. See the inner exception for details.

This was coming from EntityFramework/Internal/Validation/ValidationAttributeValidator.cs:75. Inner exception hinted of an unimplemented function.

IsValid(object value) has not been implemented by this class. The preferred entry point is GetValidationResult() and classes should override IsValid(object value, ValidationContext context).

In order to proceed, I forced a ValidationResult of Success.

Setup failed: The handle is invalid.

Reviewing the stack trace took me to EntityFramework/Core/Metadata/Edm/LightweightCodeGenerator.cs:193. Examining the code showed that CreatePropertyGetter was creating a dynamic method to map to the getter method of a property on an Entity. After some more trial and error, it became apparent that while the Type handle was mapping correctly, the RuntimeMethodHandle held an invalid value. Fortunately, the getter method on a property has the naming convention of get_PropertyName. This could come in handy to boot strap this code to allow proceeding further. Time to commit changes and review the work.

I reviewed the database that was created by the application, and I see tables being created. This means that EntityFramework is facilitating table creation which is good news. This is a good stopping point for the day.

Conclusion thus far: SqlServerCe driver won't work under Linux, however, after some heavy tweaking and commenting of EntityFramework code, SqlServer driver does work, at least as far as creating tables.

Day 1 - Porting nopCommerce to Linux

nopCommerce is an open source, feature rich, and production ready shopping cart and eCommerce solution. They are being used by a number of businesses all over the world as a platform of choice for their webstores. Out of the box, nopCommerce supports MS SQL Server and MS SQL Server Compact Edition as backend database engine. I was researching open source eCommerce solutions when I came across nopCommerce few months ago. NopCommerce peaked my interest as it is written using .Net Framework. While Mono project makes applications written on .Net framework cross platform, it lags behind .Net framework in development and lacks the latest features that .Net framework offers. When I first looked into nopCommerce, Asp.Net MVC 3 was not available for Mono and neither was Entity Framework. nopCommerce utilizes both of these features heavily. This made nopCommerce, an open source solution, not cross platform.

Recently, when the 3.0.3 development branch of Mono made both of these features available, I thought it'd be interesting to try and test Mono framework with its newly added Entity Framework and attempt to run nopCommerce under Linux.

This article will record the changes that were made to get rid of initial compile time errors for nopCommerce, run the application, and get to the Installation page.

Note: When modifying files in MonoDevelop that originate on Windows platform, and when a source control like Git is involved, it is best to not have MonoDevelop change line endings from windows to unix format to keep you commits clean.

Note: The code changes made during this experiment are available on GitHub at https://github.com/jimmy00784/nopCommerce-Linux-Mysql

The primary challenge I face is that in order to utilize Entity Framework and ASP.NET MVC 3 for .Net development, I had to upgrade the stable Mono 2.10.x to Mono 3.0.3 development release. I also had to upgrade MonoDevelop to 3.1.0 development release. This has broken the debugging capabilities from within MonoDevelop.

For porting nopCommerce, initially, I thought that simply switching the .Net Entity Framwork dlls with the Mono's dlls would be enough to run nopCommerce. I downloaded the source code from nopCommerce and loaded it in MonoDevelop. At first glance it didn't look as bad. Looking closely it becomes clear that System.Data.Entity supplied by the project would not work with Mono under Linux and there were dependency issues with EntityFramework and SqlServerCe dlls. I also downloaded the source for EntityFramework for Mono and planned to load EntityFramework as a project instead of simply adding the dll reference to the project. From earlier experience, I knew I'd be making some code changes to the EntityFramework to make it work with nopCommerce.

I added a solution folder for EntityFramework to nopCommerce solution and then added EntityFramework, EntityFramework.SqlServer, and EntityFramework.SqlServerCompact projects to it. For a quick sanity check, I compiled these new projects. EntityFramework complained that the delay sign key provided couldn't be used for assembly signing. That was easily fixed by providing the '-delaysign' compiler arguement. Recompiling didn't cause the same error but instead caused other code related errors, all of which are 'Warning as Error' which could also be fixed by unchecking the "Treat warnings as errors" settings for the project. This fix would also have to be applied to any other projects that complain in similar fashion. Also, the System.Data.SqlServerCe.dll supplied with EntityFramework.SqlServerCompact wasn't compatible with Mono on Linux. I was able to download the file from my Windows installation and that seemed to work ok for the purpose of achieving an error free compilation. The intention here was not to fix the EntityFramework project but to tighten or loosen just enough screws to run nopCommerce under Mono under Linux.

Build: 0 Errors, 54 Warnings.

That's a good place to be. Since EntityFramework project compiled, I commited the changes to the git repository so it would serve as a checkpoint in case I needed to revert any changes I make to the framework.

I replaced the EntityFramework references from the nopCommerce project and use the Entity Framework project as reference instead. Changes would be made only to Nop.Data project and the other projects under the Libraries solution folder for now. I also upgraded the projects from Mono / .Net 4.0 to .Net Framework 4.5 as that's the version EntityFramework would compile under due to other dependencies that are only available under .Net Framework 4.5. Recompiling gave only one error though enough to halt compilation of the project.

Libraries/Nop.Data/Extensions.cs(19,19): Error CS0234: The type or namespace name `Objects' does not exist in the namespace `System.Data'. Are you missing an assembly reference? (CS0234) (Nop.Data)

This was in reference to System.Data.Objects namespace needed on that class. A quick examination revealed that ObjectContext also had an issue and the only way to fix it was to include System.Data.Entity.Core.Objects in the project. This was coming from the EntityFrameworks project I included in the solution. Since I was tracking my changes in Git, I renamed all System.Data.Entity.Core to System.Data to see what it'd do.

Buid: 49 Errors, 34 Warnings.

Most of these were instances of EntityState and DbFunction references complaining for missing System.Data.Entity namespace directive. Many more errors were revealed once these errors were resolved but fortunately those required the same fix. Once all the errors were resolved, I tried replacing the references from EntityFramework.dll, System.Data.Entity.dll, and System.Web.Entity.dll with the EntityFramework project in their place. The changes made to the files under EntityFramework solution folder would only be commited to Git if replacing the Entity Framework references did not generated any namespace related errors.

As mentioned earlier, the projects under the Library solution folders would have to be upgraded to use .Net Framework 4.5 from Mono / .Net 4.0 profile since Entity Framework requires .Net Framework 4.5 features. Similarly projects under the Presentation solution folder would need to be upgraded to use .Net Framework 4.5 and references to EntityFramework.dll and System.Data.Entity.dll would need to be replaced with the project reference to EntityFramework project instead. Since all these projects compiled well without any errors, I commited the changes made to EntityFramework projects to Git.

System.InvalidOperationException

Storage scopes cannot be created when _AppStart is executing.

I had seen this before. A small change in Global.asax.cs file was needed to resolve this. Adding the following lines in the beginning of Application_BeginRequest function and including the necessary namespace references does the trick.

var ob = typeof(AspNetRequestScopeStorageProvider).Assembly.GetType("System.Web.WebPages.WebPageHttpModule").GetProperty("AppStartExecuteCompleted", BindingFlags.NonPublic | BindingFlags.Static);

ob.SetValue(null, true,null);

Giving it another run gave the following error.

System.IO.DirectoryNotFoundException

Directory '/home/karim/MonoDevelopProjects/nopCommerce_2.65_port/nopCommerce-Linux-Mysql/Presentation/Nop.Web/App_DataLocalizationInstallation' not found.

Now I was getting somewhere. Searching for the path reference in the code took me to the InstallationLocalizationService.cs file which passes the location string to webHelper.MapPath method. This took me to MapPath function in the WebHelper.cs file in Nop.Core project. The else block was written for Windows environment. This was done by the .Replace('/', '\\') method. Searching for that string pattern revealed two places total where this was used. I could check for OperatingSystem information before deciding to use the Replace function. Surrounding the code with "if(Environment.OSVersion.Platform != PlatformID.Unix)" should be sufficient. Recompiling gave no errors but running the project again did. It is a different error this time.

System.InvalidOperationException

The view 'Index' or its master was not found or no view engine supports the searched locations.

The following locations were searched:

~/Views/install/Index.aspx

~/Views/install/Index.ascx

~/Views/Shared/Index.aspx

~/Views/Shared/Index.ascx

~/Views/install/Index.cshtml

~/Views/install/Index.vbhtml

~/Views/Shared/Index.cshtml

~/Views/Shared/Index.vbhtml

At first glance it looked counterintuitive because ~/Views/Install/Index.cshtml did exist, but taking a closer look revealed that the system was looking for index with lower case "I" instead of upper case. I changed the URL in the address bar and used Install with upper case "I" and that landed me on the installation page.

After inspecting Global.asax.cs file at Application_BeginRequest, I found that one of the first thing the application does (ignoring the _AppStart fix) is EnsureDatabaseIsInstalled. Examining it revealed where the controller name install with a lower case "i" came from. I fixed the case there, recompiled and ran the application.

I now had the nopCommerce Installation Page.

Spent weekend porting NopCommerce to Linux and for MySQL. Why so excited?

Because it isn't meant to, at least not yet.

NopCommerce relies on Entity Framework which was recently open sourced by Microsoft. Mono has included it in their latest beta. It doesn't included support for MySQL database. MySQL driver for .Net has Entity Framework features turned off for Mono due to the same reasons.

What is available under Mono for Entity Framework provides more obstacles than assistance as not all features are ready and some features don't work as expected. Oh! And MonoDevelop beta that you'd use with Mono 3.0.3 beta, doesn't have functioning debugging hooks. So the code is sprawled with Console.WriteLines with nonsensical one - two word phrases to leave a bread crumb trail. But the code wasn't meant to be pretty either.

The admin portion isn't functioning yet due to an issue with generated SQL somewhere.

I'll setup a repository in GitHub soon to show the hacks that were put in place to make this possible.