Flume from C#

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.

Flume has an appender for log4j, but no support for log4net or NLog was provided.  I created a opensource appenders on GitHub.

Both of them are pretty straight forward to use with two steps:

1. Install the Nuget package


Install-Package DotNetFlumeNG.Client.NLog

2. Configure the appender

<pre><code><configuration>

  <configSections>
    <section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog" />
  </configSections>

  <nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <extensions>
      <add assembly="DotNetFlumeNG.Client.NLog" />
    </extensions>
    <targets>
      <target name="a1" type="Flume" host="yourflumeserver" port="yourflumeport" />
    </targets>
    <rules>
      <logger name="*" minLevel="Info" appendTo="a1" />
    </rules>
  </nlog>

</configuration>
</code></pre>
<code> </code>

3. That’s it!

All existing logging code will now be logged to your existing flume services!

Advertisements
Aside | Posted on by | Tagged , , , , | Leave a comment

Azure Table Storage

Azure’s Table Storage is an interesting Beast. I set out to build a sample application with it, thinking it stored blobs of JSON or unstructured classes. It does support storing rows in a signle table where different rows contain different columns, but it’s not nearly as flexible as a NoSQL DB like MongoDB, which I thought was the case. A better comparison to Azure Table Storage is Amazon’s Simple DB.

The Structure is certainly simple to use, and highly scalable, but it’s not nearly as flexible as a Document-Oriented store like MongoDB.

I’m uncertain what are best practices for designing a Azure Table Storage schema and Data Access tier. Cost of the Cloud service, such as data size or per-insertion costs are factors to account for. Also what are the ideal target applications for something a technology like this? It seems similar in some regards to Hadoop, but the Map-Reduce, Flume and other tools are not there yet.

After playing around with Azure Table Storage for a short period of time, it’s definitely easy to use and something I’ll keep in mind in future projects.

Posted in Uncategorized | Tagged , , | Leave a comment

WPF NotifyIcon must be used as x86 application

After a small configuration change, I spent a lot of time trying to figure out the following problem. In solution was simply that the WPF NotifyIcon (http://www.hardcodet.net/projects/wpf-notifyicon) requires the WPF application to be running as x86. Running as Any CPU will produce the following uninformative error:

“‘The invocation of the constructor on type ‘Hardcodet.Wpf.TaskbarNotification.TaskbarIcon’ that matches the specified binding constraints threw an exception.'”

Posted in C# | Tagged , | 1 Comment

How to make IE not suck on Windows Server 2008 R2

Anyone who has used Internet Explorer on a clean install of Windows Server 2008 R2 will have experienced the pain I’m referring to. After a default install, it’s near impossible to login to most sites or download files from public servers. The first thing I force IE to do is download and install Chrome.

Now I have an alternative – Turn off Enhanced Security!

Posted in Uncategorized | Tagged , | Leave a comment

Rosalind – Project Euler for Bioinformatics

I’m a big fan of Project Euler. It’s a great site for learning a new language and keeping your math skills sharp.

There’s now a similar site for Bioinformatics – http://rosalind.info/ that looks very cool. I’m sure I’m going to end up burning hours on it.

Posted in Uncategorized | Leave a comment

NoSQL Distilled Chapters 1-3

Here’s a couple chapters from NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence by Pramod Sadalage and Martin Fowler.

NoSQL Distilled – Chapters 1-3

Posted in Architecture | Tagged | Leave a comment

NuGet for the Enterprise

NuGet is such a great tool that it’s surprising we have gone without it for so long. Most enterprise developers will make heavy use of it to leverage external libraries. Before long most enterprise companies will see a need to share their own libraries internally. They need a private NuGet server. Thanks to the excellent guys building NuGet, it’s free and low effort to setup your own server. Below is how I did it:

  1. Install Windows Server 2008 R2
  2. Install SQL Server 2012 Express
  3. Install IIS
  4. Install .Net 4
  5. run C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis -iru
  6. Checkout code from https://github.com/NuGet/NuGetGallery.git
  7. Run build.bat
  8. Copy \PackagedWebsite to \\yourserver\c$\inetpub\wwwroot

Troubleshooting

I did run into a few snags installing it. Here are a few more steps I performed after the above installation steps.

  1. I had to configure the IIS AppPool to run as a NETWORK_SERVICE. This allows access to SQL Server.
    1. Reference: http://stackoverflow.com/questions/2251839/login-failed-for-user-nt-authority-network-servi
  2. Create & Grant IIS_Users access to a Lucene folder C:\inetpub\wwwroot\NuGet\App_Data\Lucene
  3. Create & Grant IIS_USERS access to the folder C:\inetpub\wwwroot\App_Data\Files\packages.

 

Here are the steps they give which are excellent to: https://github.com/NuGet/NuGetGallery/wiki/Hosting-the-NuGet-Gallery-Locally-in-IIS

Posted in .Net | Tagged , , | Leave a comment