MD5 Checksum with Text mode in c#

I need to develop an MD5 checksum routine to check files that were being picked up from a Unix host. I found a number of blogs/code samples that showed the basics of using the MD5CryptoServiceProvider, so I quickly put together the code.

Unfortunately when I tested the results against the sample data, my checksums were different.

After some research I narrowed down the problem to the Unix MD5Sum using the file in TEXT mode. This is native for Unix, but not windows.

Ok, I can change my routine to use text mode as well? Not quite that simple.

I could not find a way to get .NET c# to read a file in text mode.

After some more research I found that the main difference when using text mode on a windows platform is that \r\n (carriage return, line feed) have the \r removed. In binary mode this is obviously not done.

So I did a quick test and did a string replace prior to doing the MD5 hash, and voila!!! I got the match.

So here is my class I created to allow for both modes when doing the MD5 checksum.

Hope this saves someone else some time!

class MD5FileCheckSum

{

public static string CalculateChecksum(Stream inputDataStream, bool textMode)

{

using (MD5 md5Computer = new MD5CryptoServiceProvider())

{

if (textMode)

{

//in text mode - for UNIX compatibility, we need to remove the \r characters from the \r\n end of lines

//this is also the same result as using the 'c' command fopen r

StreamReader sr = new StreamReader(inputDataStream);

string textdata1 = sr.ReadToEnd();

//remove the carriage returns

string textdata2 = textdata1.Replace("\r\n", "\n");

//convert back to a byte array

byte[] textbytes = Encoding.ASCII.GetBytes(textdata2);

//do the hash

byte[] retvaltext = md5Computer.ComputeHash(textbytes);

//return the result

return BitConverter.ToString(retvaltext).Replace("-", String.Empty);

}

else

{

//binary mode - just hash the whole data

byte[] retval = md5Computer.ComputeHash(inputDataStream);

return BitConverter.ToString(retval).Replace("-", String.Empty);

}

}

}

}

SSL and Certificate security with WCF

Set up WCF to use certificates for security. There are a number of examples and tutorials, but there is still a number of gotchas in doing this, the keys we found being:

- For TRANSPORT security using Certificate authentication

• make sure you have a ROOT CA, and trust it on both the server and the client. This means the ROOT CA public key needs to be imported into the trusted certificates of both. Transport security will not work (even in dev) without a trusted CA on both the client and server.
• make sure the server SSL certificate is issued to the server name you will be using in the URL - i.e if you are using http://my.server.wcf/MyService, then issue the SSL server certificate to my.server.wcf
• make sure your client certificate has a private key and it is imported to the personal cert store on the client. Ensure the client's public key is in the trusted people on the server.
• Ensure the web site n IIS has SSL required checked on, and require client certificate checked on.
• We setup up a vanilla web site with a simple "Hello World" web page to check the SSL and client certificates are working (Require SSL and Require Client Certificate turned on). If you use IE to connect to the basic web site on the server, it should prompt you to select a certificate. If you can select your certificate and connect to a basic web site, then the same certificates will work in WCF. If not, then it wont work in WCF either!
• you can use certificate mapping to map to a windows account, and then use .net security demands in your application is normal. Make sure you set windows account mapping on in the wcf server configuration (mapClientCertificateToWindowsAccount="true") AND in IIS.
• you can achieve this with self signed certificates or CA assigned certificates
• you can achieve this in IIS 6 and IIS 7

- For MESSAGE security using certificates

• basic Message security with certificates is simpler
• use PeerTrust, and trust the client public key on the server, and the server public key on the client (Trusted People).
• No need for a trusted CA (or any CA) in this configuration
• use the identity/dns settings in the wcf configuration to match the certificate names issued to the server and client.
• certificate mapping to windows accounts does not work using message security
• you can achieve this with self signed certificates or CA assigned certificates
• you can achieve this in IIS 6 and IIS 7

Thats a simple checklist of things to look out for. Hope it helps someone.

Create ADAM accounts in VB.NET SSIS script

Had to write an SSIS script to Load accounts in to ADAM and had trouble trying to get it working. In the end it looks pretty simple but the gotcha's are in what you can and can not set against the user class, and also setting the password encoding and port.

Here is my method that met my requirements for creating the ADAM accounts with a password, ready to use. I searched all over for how to do this and borrowed bits and pieces from other posts and samples, but in the end did not find a sample that was as simple as this with only one commit.

NOTE this is for a NON-SSL installation, where the ADAM accounts are used by an ASP.NET application via the Membership Provider.

Also note that in this installation the user name and email are required to be the same - hence the search on the mail property being equal to the user account to ensure duplicates are not created.

---------------------------------------------------------------------------------------------

Public Function CreateADAMAccount(ByVal LdapDomain As String, ByVal AUser As String, ByVal APwd As String, ByVal userAccountToCreate As String) As Boolean

Dim resList As List(Of String) = New List(Of String)()

Dim distinguishedName As String = String.Empty

Dim connectionPrefix As String = LdapDomain

Dim entry As DirectoryEntry = New DirectoryEntry(connectionPrefix, AUser, APwd, AuthenticationTypes.ServerBind)

Dim mySearcher As DirectorySearcher = New DirectorySearcher(entry)

' do the search just to be sure that there is no other account with that email address

mySearcher.Filter = "(&(objectClass=user)(mail=" & userAccountToCreate & "))"

mySearcher.PropertiesToLoad.Add("distinguishedName")

mySearcher.PropertiesToLoad.Add("cn")

mySearcher.PropertiesToLoad.Add("name")

Dim results As SearchResultCollection = mySearcher.FindAll()

Dim res As Boolean = False

If results.Count > 0 Then

res = False

Else

Dim usr1 As DirectoryEntry = entry.Children.Add("CN=" & userAccountToCreate, "user")

usr1.Properties("userPrincipalName").Value = userAccountToCreate

usr1.Properties("mail").Value = userAccountToCreate

usr1.Properties("userPassword").Value = "XXXXXXXXX"

usr1.Properties("msDS-UserAccountDisabled").Value = False

usr1.Properties("passwordQuestion").Value = "XXXXXXXX"

usr1.Properties("passwordAnswer").Value = "XXXXXXXX"

usr1.Options.PasswordEncoding = PasswordEncodingMethod.PasswordEncodingClear

usr1.Options.PasswordPort = 389

usr1.CommitChanges()

res = True

End If

entry.Close()

entry.Dispose()

mySearcher.Dispose()

Return res

End Function

--------------------------------------------------------------------------------------------

In coming to this solution I tried to use the Membership Provider within SSIS - dynamically setting the configuration settings etc. However I came to the conclusion while testing that it was not going to work as the non-ssl install required the machine key to be available (for encryption) in the system.web section of the configuration - not likely in an SSIS setup. This was getting to be too much of a hack so I reverted to the directory services solution and ended up with the above.

Parameter Manipulation attack on ASP.NET MVC web site

Most of the ASP.NET MVC web site security patterns I have seen are focused on stopping non-authenticated users from accessing the site, or non-authorised users from accessing methods.

These methods work really well to secure the site.

However, in developing an ASP.NET MVC web site I learned the hard way that the patterns in tutorial and training materials for ASP.NET MVC are prone to parameter manipulation attacks.

Luckily for me it was identified during a pen test, and could be fixed.

The problem is that most of the examples you see rely on passing record keys to methods. The user is authenticated, and is authorised to use the methods. The problem occurs when the user decides to manipulate the record keys - this calling a method with a record key that you did not provide them.

For example, if you have an EditUser method that accepts a user id as a key parameter. The method is called by a button on a page (say a search page), that posts the form with the key value in a field. The malicious user can save the web page to a local file, modify the key value, load the page into the current sessions browser window, and submit the form.

The anti-forgery token is valid, the user is valid, the user is authorised - however they are now viewing or editing a user they were not supposed to.

The ASP.NET MVC developer needs to be aware of this vulnerability, and re-check that the user has rights to the records they are viewing or editing when the parameters are returned.

This can be done by performing the authorisation check again, storing the valid values in session state, encrypting keys, or any other method you care to use, however you can not trust that the parameter has not been tampered with when you receive it!

BEWARE Reference variables in LINQ Lambda Expressions

I have come across a 'feature' of LINQ and lambda expressions where the variables used in the lambda are stored by reference, and changing them before you are finished with the result set cause the result set to change on the fly!

Now I am not across the inner workings of LINQ and lambda, however the results from doing the following code are not what I would have expected.

private void button1_Click(object sender, EventArgs e)
{
System.Diagnostics.Debug.WriteLine("Testing:" + DateTime.Now.ToString());
List<Dictionary<int, int>> ld = new List<Dictionary<int, int>>();

ld.Add(new Dictionary<int, int>() { { 1, 10 }, { 2, 20 }, { 3, 30 } });
ld.Add(new Dictionary<int, int>() { { 1, 20 }, { 2, 300000}, { 3, 40000000 }, { 4, 500000000 }});

int indx = 10;

var ds = ld.Where(itm => itm[1] == indx);

if (ds.Count() > 0)
{
System.Diagnostics.Debug.WriteLine("Count=" + ds.First().Count());
indx = ds.First()[2]; //this is a value I want to keep for later
int valueIwant = ds.First()[3]; //this is the value I want from this result

System.Diagnostics.Debug.WriteLine("Value I Want=" + valueIwant);
//oh no ??? that's not the value I wanted ???
System.Diagnostics.Debug.WriteLine("Count=" + ds.First().Count());
//and now the count changed ???
}

}

I would expect the LINQ query to return the first row, (where itm[key==1] == 10).

And valueIwant to equal the third value in that result dictionary (itm[key==3]).

The first count confirms we are working with row 1 with 3 records in the result dictionary.

HOWEVER, changing the indx value prior to getting valueIwant causes the result to change dynamically - thus returning a new result set, which in this case is row 2.

The end result in my variable is 40000000 - big diff. And the final count is 4, as it is the second record now.

This is a real trap - especially when using LINQ/lambda to work down a hierarchy of values.

My suggestion would be to use a pattern where any lambda expressions that use a variable should have the variable assigned to a lambda specific variable prior to creating the expression.

i.e.

int indx = 10;

int tmpidx = intidx;

var ds = ld.Where(itm => itm[1] == tmpidx);

if (ds.Count() > 0)

{

indx = ds.First()[2]; //this is a value I want to keep for later

int valueIwant = ds.First()[3]; //now I get what I wanted !!!!

I can't determine if this is by-design, a feature to support delegates, or a genuine bug, but it is definately something to watch out for!

Update: The legends on the MSDN forum suggest using .ToList() on the end of the Where statement to generate a fixed list result - thus separating the result set from the object query. It is also noted that each call to Count() and First() will cause the expression to re-evaluate!

http://social.msdn.microsoft.com/Forums/en-US/linqprojectgeneral/thread/b1480b00-44ec-4b32-984f-cdd53d87e99d

Telstra MF636 3G modem in Windows 7

When I tried to get the Telstra 7 Series MF636 modem working in Windows 7 I found that it would load the connection manager software but fail to find the sim/modem.

To get the modem to work I had to use the 'Safely Remove Hardware...' option to eject the ZTE modem which was registered as a USB drive as well.

Once the USB drive was ejected, the connection manager would then take around 5 to 10 seconds to recognise the sim and modem and then everything works fine.

Best feature of Windows 7 ???

Recently setup W7 on my lappy (Toshiba P10).

Installation is quite simple, however getting the drivers (most of which are BETA) is a bit painful.

The best feature I have found so far is the mouse-over on the active windows on the task bar.

I dont recall Vista having anything like this?

When you mouse over an application the task bar, the thumbnails of the windows popup, and when you mouse over one of the thumbnails, all the other windows become transparent except for the moused-over application's window.

I think that is a great feature!

Using ROWNUMBER() to sort views!

Had a bit of an epiphany today when creating a view.

I needed to be able to join the view back on itself to create a running total column. In order to do this I thought I would use ROWNUMBER() to create a key and join on values less than that key.

In doing so I realised that ROWNUMBER() allows you to create a sorted view. This is something that anyone who has tried to create a sorted view will know can not be done unless you use a SELECT TOP statement, which is cludgey.

Using a " SELECT ROW_NUMBER() OVER(ORDER BY field1, field2, field3) AS 'ROWNUMBER', field1, field2, field3 ... FROM ... WHERE ... " statement however has the effect of sorting the values and you can use this to create a sorted view.

i.e.

CREATE VIEW vwTestView AS

SELECT ROW_NUMBER() OVER(ORDER BY field1, field2, field3) AS 'ROWNUMBER', field1, field2, field3 ... FROM ...

WHERE ...

Obviously this is only good if you are able to put the field in the view without upsetting anybody. But if you can it works well.

mailto: link in a HTML Popup

Today I had to use a HTML Popup to display some person details, including and email address.

To get the email link to launch a new email window I thought I could just insert the mailto: markup into the anchor in the popup body, however it did not respond.

I found that to get this to work I needed to insert my popup content into a DIV, pass the DIV content as the content for the popup body, and have the onclick event on the mail address anchor make a call to get the parent window to navigate to the mailto: URL.

All a bit cludgey, but it worked in the end. The resulting HTML looked like this:

<DIV id="data1" style="display:none">

<SPAN onclick="javascript:parent.window.navigate('mailto:xxxxxx@xxxx.xxx.xx')">

<B>Email:</B><a href="#" >xxxxxx@xxxx.xxx.xx</a></SPAN>

<BR> ... more HTML ...

</DIV>

<a href="#" onclick="javascript:

var oPopup = window.createPopup();

var oPopBody = oPopup.document.body;

oPopBody.style.backgroundColor = 'lightyellow';

oPopBody.style.border = 'solid black 1px';

oPopBody.innerHTML = data1.innerHTML;

oPopup.show(0, 16, 350, 75, event.srcElement);" style="text-decoration:none" >...</a>

Recently had to get a Telstra 21 3g card working with a Netcomm 3G wireless router N3G002W.

There were some hassles getting it all configured but the in the end it DOES WORK!

So the things to look for are:

• Dont turn off the radio when you disconnect the dongle from your pc. I just pulled it out to make sure. The option in the bigpond connection manager app to close and exit I think turns of the radio.
• Press and hold the SHIFT key when clicking on the OPTIONS in the telstra bigpond connection manager app. This gives you some extra tabs in the options dialog. Click the checkbox in one of the new tabs that says that the device is already registered.
• use telstra.bigpond as the APN, and put in the username and password. Other default settings worked fine.

Did not really see any speed increase over the 7.2 where I was though (country area) !