Crafting a C# Forms Editor from scratch

7:28:00 am 1 Comments

Sample Image - SharpFormEditorDemo.jpg

Contents

Introduction

A Forms Editor allows you to add, move, resize, and delete controls on your form. The dialog editor of VC6 and the Forms Designer of VS.NET are Forms Editors we commonly use at design time.
In the .NET framework, this functionality is wrapped as several services, to enable programmers to write their own Forms Editors that create forms in a manner similar to the way used in the Forms Designer of VS.NET. By relying on the .NET framework, the programmer doesn't need to care about how to select/move/resize a control and draw the selection rectangle. He only needs to implement some base services, and the framework will do the control selection and draw the selection effect.
This article tries to show you how to write a Forms Editor without implementing the base services, that is, performing the control selection and drawing the selection effect by yourself. I hope, from this, you can deduce what's happening under the covers when you rely on the .NET framework to implement your own Forms Editor.

Approaches to doing Form edit

There are two major tasks that needs to be performed when you edit a control on a form:
  1. Prevent the controls in the container form from accepting the system events, such as mouse click and key press.
  2. Paint the selection rectangle "above" the other controls in the container form when the selected controls are being moved across other controls (see the demo picture).
The second item is more annoying because Windows controls are painted by the OS after you paint the selection rectangle on the surface of the container form. So if the selection rectangle is overlapped with another control in the container form, it will always be covered by the control.
The following projects solve the above two problems from different aspects:
  1. Johan Rosengren's DialogEditorDemo. This is an excellent dialog editor application that has many other features. It doesn't put real Windows controls on the dialog. Actually, it just draws the GDI objects that look like Windows controls on the surface of the dialog, and then draws the selection rectangles. Since the painting of the pseudo controls and the selection rectangles are all performed by the user application, it's easy for you to paint a selection rectangle "above" the pseudo control when they are overlapped. Johan's application is an MFC project, I've tested his approach in the .NET environment with C#, and it works pretty fine with simple controls, because the .NET framework has already implemented several control painting methods such as DrawButton(), DrawCheckBox(), DrawComboButton(), and DrawRadioButton() in System.Windows.Forms.ControlPaint. However, this approach has its limitations, it would be difficult to paint other complex controls, and because the painted controls are not real Windows controls, you can not edit their properties at runtime.
  2. Nashcontrol's C# Rect Tracker. This approach partially solves the above mentioned two problems by implementing the selection rectangle tracker (RectTracker) as a UserControl. By doing so, the RectTraker can be brought to the front as a UserControl and be painted by the OS "above" other controls that has a lower Z-Order. But the effect is not satisfactory, sometimes moving/resizing a control does not work as you expect.
  3. SharpDevelop. It is an open source IDE for the .NET platform. As I said above, in the .NET platform, the functionality to select/move/resize a control is already wrapped into the .NET framework; all you have to do is to implement the interfaces and the base services such as IDesignerHost, IContainer, IToolboxService, IUIService, and ISelectionService. But this approach doesn't show you what is under the hood since the framework does the tough part for you.

How does Visual Studio do it?

The answer is, draws the selection rectangle on a transparent overlay on top of the container form. When you explore a form in the Forms Designer of Visual Studio with Spy++, you'll find there is a transparent control called "SelectionUIOverlay" and it is just above the container form that is being edited.
ExploreWithSpy++

Implementing your own RectTracker and SelectionUIOverlay

What we need now is a RectTracker to track the selected objects, and a truly transparent SelectionUIOverlay on which to draw the selection rectangle.

C# RectTracker

The CRectTracker class of MFC has existed for a long time, it was always used as a tracker for rectangle objects. You can find its definition in "AFXEXT.h", and its source code in "...\atlmfc\src\mfc\trckrect.cpp".
Since the .NET framework doesn't wrap all the Windows APIs, to port the class to C#, you still need to call some API functions through System.Runtime.InteropServices. However, you can draw a rubber band (a focus rectangle that tracks with the mouse pointer while you hold down the left mouse button) and a dragged rectangle by using ControlPaint.DrawReversibleLine or ControlPaint.DrawReversibleFrame, this can avoid calling the GDI+ APIs.
//

private void DrawDragRect(Graphics gs,Rectangle rect,Rectangle rectLast)
{
   ControlPaint.DrawReversibleFrame(rectLast,Color.White,FrameStyle.Dashed);
   ControlPaint.DrawReversibleFrame(rect,Color.White,FrameStyle.Dashed);

}
I've also derived a FormRectTracker from the C# RectTracker to select and resize the container form. It simply overrides the HitTesthanles() method of RectTracker to prevent the container form from being moved or resized by dragging on its top/left boundaries.
//

protected override TrackerHit HitTestHandles(System.Drawing.Point point)
{
       TrackerHit hit = base.HitTestHandles (point);
       switch(hit) {
              case TrackerHit.hitTopLeft:
              case TrackerHit.hitTop:
              case TrackerHit.hitLeft:
              case TrackerHit.hitTopRight:
              case TrackerHit.hitBottomLeft:
              case TrackerHit.hitMiddle:
                    hit = TrackerHit.hitNothing;
                    break;
              default:
                    break;
       }
       return hit;
}
During my porting of CRectTracker from MFC to C#, I found fanjunxing's implementation here on the internet. Since both implementations are all statement to statement translations from the same MFC class to C#, most parts of the two implementations are very similar. The difference lies in how to draw a rubber band and reconstruct a Rectangle object when you resize the selection rectangle.

Transparent SelectionUIOverlay

Because the .NET framework doesn't support transparent controls directly, you may think that we can use a transparent sub-form as a transparent control by setting the TransparencyKey to the value of the its BackColor. Unfortunately, this approach doesn't work. If the parent form of a sub-form is not transparent, setting the sub-form's TransparencyKey to its BackColor won't make the sub-form transparent.
To implement a transparent control, you need to:
  1. Add the transparent style to the control window:
    //
    
    protected override CreateParams CreateParams 
    { 
           get 
           { 
                  CreateParams cp=base.CreateParams; 
                  cp.ExStyle|=0x00000020; //WS_EX_TRANSPARENT 
    
                  return cp; 
    
           } 
    }
  2. Override the OnPaintBackground event. This is necessary to prevent the background from being painted.
    protected override void OnPaintBackground(PaintEventArgs pevent) 
    { 
           //do nothing 
    
    }
  3. Write a message filter that implements the IMessageFilter interface. This is the most tricky part of a truly transparent control. If you want to draw something on the transparent control, and at the meantime move/resize the controls beneath it, you can write a message filter to prevent the controls from refreshing, and then expand the transparent control's Invalidate() function to draw your own item and the other controls beneath it.
    public class MessageFilter:System.Windows.Forms.IMessageFilter
    {
           ...
           public bool PreFilterMessage(ref System.Windows.Forms.Message m)
           {
                  Debug.Assert(frmMain != null);
                  Debug.Assert(frmCtrlContainer != null);
    
                  Control ctrl= Control)Control.FromHandle(m.HWnd);
                  if(frmCtrlContainer.Contains(ctrl) && m.Msg == WM_PAINT)
                  {
                        //let the main form redraw other sub forms, controls
    
                        frmMain.Refresh();
                        //prevent the controls from being painted
    
                        return true;
                  }
                  return false;
           }
    }
    //
    
    public class SelectionUIOverlay : System.Windows.Forms.Control
    {
           ...
           private void InvalidateEx() 
           {
                 Invalidate();
                 //let parent redraw the background
    
                 if(Parent==null) 
                       return;   
                 Rectangle rc=new Rectangle(this.Location,this.Size); 
                 Parent.Invalidate(rc,true);
                 ...
                 //move and refresh the controls here
    
           }
    }

Add a control at runtime

With C#, you can create an object with the "new" operator, or you can create it through reflection. I'm wondering if the .NET CLR just uses the same mechanism to treat these two approaches. I noticed that only when you run the application from the debugger, there seems to be a minor performance issue with reflection. That's why I add a switch statement and some "new" operators in the demo code. But if you run the application directly, there seems to be no performance issues with reflection at all, no matter it's a debug or a release version.
//

public static Control CreateControl(string ctrlName,string partialName)
{
       Control ctrl;
       switch(ctrlName) 
       {
             case "Label":
                    ctrl = new Label();
                    break;
             case "TextBox":
                    ...
             default:
                    Assembly controlAsm =
                            Assembly.LoadWithPartialName(partialName);
                    Type controlType = 
                            controlAsm.GetType(partialName + "." + ctrlName);
                    ctrl = (Control)Activator.CreateInstance(controlType);
                    break;
       }
       return ctrl;

}

Edit a control at runtime

Since the .NET framework already has a PropertyGrid control to do runtime property edits, it's fairly easy to edit a control's properties after you create it. All you have to do is set the PropertyGrid's SelectedObject property to the selected control.
The transparent SelectionUIOverlay is put on top of the container form, so it can naturally prevent the container form and the controls in the container from receiving mouse click and keyboard events. Another thing you need to do is to pass the Graphics of the transparent control to the RectTracker when invalidated, and the RectTracker will then draw the selection rectangle with it.

Copy & Paste a control

I've posted another article to present a tentative solution, which you can find here. I tried to copy & paste a Windows Forms control through serializing its properties. However, this approach has its limitations, and needs extra handling of the TreeView control and the ListView control.

About the IDE

The demo application uses luo wei fen's excellent docking library, WinFormsUI, and Aju.George's ToolBox class. I made some minor modifications with the ToolBox class to adapt it to the WinFormsUI framework.

Final words

I decided to write this article not because I wanted to give some classes as replacements for the corresponding functionalities wrapped by the .NET framework. My intention is to illustrate some basic principles on how to write a Forms editor and give a concrete implementation of it. I think the implementation of the C# RectTracker class and the transparent control can also be applied to other circumstances, such as shape edit and picture/diagram edit.

Download source and executable - 201 Kb

1 comments:

Sending and playing microphone audio over network

7:25:00 am 0 Comments

Screenshot - view.jpg

Introduction

This example shows you how to receive data from a microphone and stream it over UDP to another computer. The example application can act like a direct phone, if both endpoints listen for data and send microphone data to each other. One would probably suspect that no source code exists for that, but of course it does. I hate those who will do commercial advertising. There is also a second related project what will contain a UDP server that we need to send/receive audio and compress it with g711 codec.Though only UDP is not the best way to transport audio data, RTP is the right way to go. RTP adds quality of service to transported audio, you can see how many packets are lost and can even arrange disordered packets. I will try to add an RTP example soon, so be calm, it's under way. There are some similar example applications, but most of them aren't well commented and missing some parts, so I will try to fill this part.
The package contains:
  • LumiSoft.Media - Audio related API (Included in example app)
  • LumiSoft.Net - UDP server, G711 codec
  • Example Application

Using the code

  • WaveIn - class provides a simple way to receive audio from a microphone.
    Actually all what you need to do is:
    WavIn.Devices - returns all available input devices from where we can get data.
    /// <summary>
    
    /// Application main class.
    
    /// </summary>
    
    public class Test
    {
        private WavIn m_pSoundReceiver = null;
    
        /// <summary>
    
        /// Default constructor.
    
        /// </summary>
    
        public Test()
        {
            // G711 needs 8KHZ 16 bit 1 channel audio, 
    
            // 400kb buffer gives us 25ms audio frame.
    
            m_pSoundReceiver = new WavIn(WavIn.Devices[0],8000,16,1,400);
            m_pSoundReceiver.BufferFull += new BufferFullHandler 
                                             (m_pSoundReceiver_BufferFull);
            m_pSoundReceiver.Start();
        }
    
        /// <summary>
    
        /// This method is called when recording buffer is full 
    
        /// and we need to process it.
    
        /// </summary>
    
        /// <param name="buffer">Recorded data.</param>
    
        private void m_pSoundReceiver_BufferFull(byte[] buffer)
        {
            // Just store audio data or stream it over the network ... 
    
        }
    }
    
  • WaveOut - class provides methods for playing out streaming data.
    The only thing you need to do is just call waveoutInstance.Play method.
    In my opinion, the whole example application has been coded well enough, so dig into the code. Note: Sound quality depends on network delay jittering, if there will be too big a variance in delays, voice will have holes in it. In addition, UDP packet loss and disordered packets will affect it too.



0 comments:

Disconnected Client Architecture

7:05:00 am 0 Comments

A look at an offline client architecture that I've implemented in an application for a client.

Introduction

This article discusses a disconnected client architecture that I recently added to a commercial product that I'm developing. Readers should be aware of the Smart Client Offline Application Block. The architecture that I present in this article has similarities. Because my product already has a rich architecture for communication and transaction management, I chose an implementation that works closely with that architecture. You can read about various pieces of that architecture in the following articles:
DataTable Transaction Logger
DataTable Synchronization Manager
Compressed, Encrypted Network Stream
Job Queue
Simplest Tcp Server

Regarding The Download

The download for this article is not a complete demo application. It's more like an electronic kit with a PC board and components, and you need to provide the soldering iron, solder and labor to put it together. The primary purpose here is to discuss architecture rather than implementation, so the download consists of the components that you might find useful to create your own client apps with offline capabilities.

Offline Challenges

Microsoft's Offline Application Block (OAB) poses some questions regarding offline challenges, and they seem like a good starting point to discuss the architecture that I developed.

How does the application determine if it is online or offline?

There are three places where an application determines that it is offline:
  1. During the attempt to connect to the server
  2. An exception being thrown while sending or receiving data
  3. An exception being thrown while in a wait state waiting to read data

During The Attempt To Connect To The Server

The Connect method illustrates the implementation that determines when the connection attempt fails:
public override void Connect()
{
  // If we have offline transactions, reconnecting is going to have to be

  //  done in a completely different way.

  // If the connection hasn't been created...

  if (tcpClient == null)
  {
    tcpClient = new TcpClient(); // Create a TCP client.


    try
    {
      tcpClient.Connect(host, port); // Connect.

    }
    catch (Exception)
    {
      // Let API handle connection failure.

      RaiseConnectionFailed();
      tcpClient = null;
    }

    // Only continue if connection succeeded.

    if (tcpClient != null)
    {
      InitializeReader();
    }
  }
}
If the connection fails, the ConnectionFailed event is raised. Typically, the event handler switches the client into a disconnected state:
protected void OnConnectionFailed(object sender, EventArgs e)
{
  // If disconnected operation is not allowed, throw an exception.

  if (!allowDisconnectedOperation)
  {
    throw new ConnectionException("Connection with the server failed.");
  }

  if (isConnected)
  {
    SetDisconnectedState();
    disconnectedServerComm.HandleConnectionFailure();
    connectedServerComm.StartReconnectThread();
  }
}
A thread to attempt reconnection is also started.
protected void ReconnectThread()
{
  bool connected = false;

  while (!connected)
  {
    // try every second.

    Thread.Sleep(1000);

    try
    {
      tcpClient = new TcpClient(); // Create a TCP client.

      tcpClient.Connect(host, port); // Connect.

      connected = true;
      // Success!

      RaiseReconnectedToServer();
    }
    catch
    {
      // try again.

    }
  }
}
The ConnectedToServer event is raised when the client successfully reconnects.
Thread Issues
The ConnectedToServer event is raised in a worker thread. This is an important issue because this event will be raised asynchronously. The event handler and methods that it calls must be thread safe. I use a specific object to block execution of communication and the reconnect process to ensure the smooth transition from a disconnected state to a connected state:
void OnReconnectedToServer(object sender, EventArgs e)
{
  // Block all command/responses until we're done here. Wait until a 

  // current command/response

  // is completed before entering here.

  lock (commLock)
  {
    ...
The commLock object:
protected object commLock = new Object();

public object CommLock
{
  get { return commLock; }
}
is used during all communications with the server. There is a single method entry point for sending a command to the server and receiving the response (incidentally, the "command" is a synchronous process--a response must be received before processing continues):
public static class IssueCommand<T> where T : IResponse, new()
{
  /// <summary>

  /// Issue the command and receive the response.

  /// </summary>

  /// <param name="api">The api is required in case the server goes down 

  /// and the ServerComm instance switches to the disconnected server comm 

  /// instance.</param>

  /// <param name="cmd">The command to issue.</param>

  /// <returns></returns>

  public static T Go(API api, ICommand cmd) 
  {
    T resp = new T();

    lock (api.CommLock)
    {
      api.ServerComm.Connect();
      api.ServerComm.WriteCommand(cmd);
      api.ServerComm.ReadResponse(cmd, resp);
    }

    return resp;
  }
}
The above method provides a single entry point for all communications with the server, allowing synchronization with the asynchronous reconnected event.
Generics
Generics are used to facilitate deserializing the correct response. Without generics, the caller would need to cast the the return response. This isn't a big issue, but I feel it improves the robustness of the code to specify the response class, for example:
ICommand cmd = new LoginCommand(username, password);
LoginResponse resp = IssueCommand<LoginResponse>.Go(this, cmd);
This ensures that resp is the same type. In other implementations, you may, for example, put information about the response type in the command. This would actually be even more robust, since there would be no possibility of accidentally specifying the wrong response type.

An Exception Being Thrown While Sending/Receiving Data

A TcpLibException (this is my own exception) is raised by the communication service when an exception occurs while writing data. The WriteCommand method raises the CommandFailed event so that the client has the chance to handle the command in a disconnected state:
public override void WriteCommand(ICommand cmd)
{
  try
  {
    comm.BeginWrite();
    CommandHeader hdr = new CommandHeader(sessionID, cmd.CommandId);
    comm.WriteData(hdr); // Write the header.

    cmd.Serialize(comm); // Write the command data.

    comm.EndWrite();
  }
  catch (TcpLibException)
  {
    RaiseCommandFailed(cmd);
  }
}

An Exception Being Thrown While Waiting For Data

The reader thread blocks until data is available. The communication service raises a TcpLibException if the connection with the server is lost. The reader thread handles this exception and raises the ConnectionFailed event:
while (!stopReader)
{
  try
  {
    // Start the read.

    comm.BeginRead();
    ResponseHeader respHdr;
    // Read the response header. This blocks until an exception or the 

    // response header is read.

    respHdr = (ResponseHeader)comm.ReadData(typeof(ResponseHeader)); 
    // Get the appropriate response instance.

    IResponse resp = (IResponse)Activator.CreateInstance(
         responseTypes[respHdr.responseId]);
    // Read the actual response.

    resp.Deserialize(comm);
    // Done reading.

    comm.EndRead();

    // If this is actually a notification...

    if (resp is SyncViewResponse)
    {
      // Queue the notification job so the data gets sync'd separately

      // from this thread.

      SyncViewResponse svr = (SyncViewResponse)resp;
      syncQueue.QueueForWork(svr);
    }
    else
    {
      // Otherwise queue the response.

      lock (responseData)
      {
        responseData.Enqueue(resp);
      }
    }
  }
  catch (TcpLibException e)
  {
    // If this is not an exception resulting from a controlled close of 

    // the connection...

    if (!stopReader)
    {
      // Force a disconnect.

      Disconnect();
      // Terminate the reader.

      stopReader = true;

      // And enqueue a client error.

      lock (responseData)
      {
        responseData.Enqueue(new ConnectionErrorResponse(e.Message, 
          e.StackTrace));
        RaiseConnectionFailed();
      }
    }
  }...
Thread Issues
Here the ConnectionFailed event can be raised by:
  • a failure when establishing the connection (usually the main thread)
  • a failure to write a command to the server (usually the main thread, but could be a worker thread as well)
  • a failure to read the response because the connection was lost (the reader thread)
Therefore, the ConnectionFailed event handler must take into account that it can be called from different thread context.

If the connection can change at unpredictable times, how should the application components that depend upon the connection state be notified?


Ideally, the user should continue working with the application without even knowing that the server went down. This is critical requirement for some of my clients because the client application is not something the end user directly interacts with (via a traditional UI, keyboard and mouse). Other client apps can be running processes that may take days to complete but communicate to the server frequently for additional job assignments and report current job status. Even with a UI-based client application, the idea is to handle the state change transparently.
This is achieved by having:
  • a specific and small set of commands and associated responses that the client can send to the server and receive back
  • using a common interface for command and response unifies read/write methods and serialization
  • a single point of entry for issuing a command to the server and receiving the response
Typically, the only application component requiring notification is the API layer, which switches from a connected state to a disconnected state:
protected void SetDisconnectedState()
{
  lock (commLock)
  {
    if (isConnected)
    {
      isConnected = false;
      serverComm = disconnectedServerComm;
    }
  }
}

How and where should the application store data locally so that it can be accessed while offline?


The client application stores data views as a snaphot in a discrete file.

DataView Snapshots

The client works exclusively with discrete DataView instances provided by the server. These are cached locally using the compression and encryption technology that I described in my article Raw Serialization, and utilize the raw serializer described in my article xxx. So, for example, to write out a DataView involves the public method:
public static void Write(DataView dv, string name, string prefix)
{
  StreamInfo streamInfo=InitializeSerializer(key, iv);
  RawDataTable.Serialize(streamInfo.Serializer, dv.Table);
  EndWrite(streamInfo);
  WriteToFile(prefix + "-" + name + ".cache", streamInfo.WriteBuffer);
  // Do last so memory stream isn't closed.

  streamInfo.EncStream.Close();
}
Initializing the serialization stream:
protected static StreamInfo InitializeSerializer(byte[] key, byte[] iv)
{
  MemoryStream writeBuffer = new MemoryStream();
  EncryptTransformer et = new EncryptTransformer(EncryptionAlgorithm.Rijndael);
  et.IV = iv;
  ICryptoTransform ict = et.GetCryptoServiceProvider(key);
  CryptoStream encStream = new CryptoStream(writeBuffer, ict, 
      CryptoStreamMode.Write);
  GZipStream comp = new GZipStream(encStream, CompressionMode.Compress, true);
  RawSerializer serializer = new RawSerializer(comp);
  StreamInfo streamInfo = new StreamInfo(encStream, comp, 
      writeBuffer, serializer);
  streamInfo.Iv = et.IV;
  streamInfo.Key = et.Key;

  return streamInfo;
}
And writing the data out to a file:
protected static void WriteToFile(string fn, MemoryStream ms)
{
  FileStream fs = new FileStream(fn, FileMode.Create);
  BinaryWriter bw = new BinaryWriter(fs);
  int len = (int)ms.Length;
  bw.Write(len);
  bw.Write(ms.GetBuffer(), 0, len);
  bw.Close();
  fs.Close();
}
Technically, I could probably have attached the FileStream to the serializer rather than a MemoryStream.

Can that data become stale?

The data can become stale if it is older than another update that is done by another client. There is an implicit assumption though that newer data is more accurate. When synchronizing with server, the question for the server becomes, is the data I'm getting from the client stale, meaning that some other client has already updated the record more recently? ***

When should it be refreshed?

The persistent store is synchronized when the client reconnects to the server, and the client is synchronized with a new snapshot of the view after the server is synchronized. Generally speaking, this approach works well and will immediately update the user's view of the data. The complexity here is that this may require a client-side business rule to deal with changes that have occurred in the new view. For example, I use a notification service to inform the client as to state changes in an alarm record. When the manager clears an alarm at his station (which actually updates a row in the database), this automatically sends a notification to the appropriate client to clear the alarm flag in the corresponding client hardware. If the client is disconnected, this notification is not issued. Instead, when the client is resynchronized, a business rule must fire that compares old data with new data to determine what alarm flags, if any, need to be cleared.

Should the application behave differently when it does not have access to all the requisite data or services?

To the maximum extent possible, no, it should not act differently. I have endeavored to ensure that this is achieved. There are several areas that cause difficulties.

Custom SQL Statements

My product supports custom client-side SQL statements that can be used in workflows or called directly through the client API layer. For an offline application, I don't support custom SQL statements. At some point these might be able to be run on the client but ideally, for any offline situation, custom SQL statements need to be avoided.

Reports

Reports require querying the server to either generate the report at the server or to get the DataSet necessary to generate the report at the client. When offline, reports are not available.

Monitoring And Realtime Notifications

Besides the client behaving differently, the enterprise may be monitoring whether the clients themselves are offline even though the server appears to be online. When offline, realtime notifications such as alarms, income, sensor and hardware status are not possible. This might be a critical enough issue to the enterprise that other mechanisms for notification might be needed when offline. Dealing with offline clients doesn't just involve how the client responds but may also affect how a monitoring application reports the offline client.

How and where should transactional data (message data) be stored while the application is offline?


There's really two parts to this question. Should the server implement a transaction mechanism to update offline clients when they become online, and how does the client manage offline transactions?

Server Transactions--Good Or Bad?

Well, there is no good or bad answer. When designing my product, I made the decision that the server would not maintain a transaction queue. Server-side transaction queues add a lot of complexity. How large do you let the transaction queue get before flushing it and requiring a fresh reload of the view? How do you track the positions that different clients may be at within the transaction queue when they connect to the server? When an offline client synchronizes the server, are you making sure that the server does not end up re-synchronizing that particular client? How does restarting the server affect synchronization when the transaction queue is maintained in memory? If a row is deleted, do you go through the transaction queue and delete transactions associated with the deleted row? What if there were client-side business rules that trigger on those transactions, that might still need to run? Similarly, if a field is updated, do you delete previous transaction updates? How scalable is the server architecture when it's maintaining a transaction queue?
Yes, I could go on and on. None of these questions have right answers, and sometimes the answer is so application specific that it seemed to me that maintaining a transaction queue at the server was actually bad. On the other hand, the "good" architecture now requires that the client gets a complete snapshot from the server whenever it requests a view. Potentially, the client could utilize a cached view and just get the synchronizing transactions. Perhaps this would be less data, faster, and less of a burden on the persistence server. Again, questions that cannot be answered generically with the expectation that the implementation will meet the application specific needs. So in the end it was the KISS approach that won the day, not the pro or con arguments for one implementation or another.

DataView Transactions

Ironically, after looking at server side transactions, you will realize that DataView transactions are managed at the client! In order to support an offline client, the client must record transactions not only when offline, but also when online, until the data view is reloaded. The following sequence diagram illustrates the different modes and how client-local transactions are managed.
When the client is online:

The client:
  • Connects to the server
  • For a given view, gets offline transactions associated with a view
  • Posts them to the server
  • Loads the current view from the server, obtaining a current snapshot of the view
  • Saves the view to the local cache
  • Deletes the offline and online transactions. The view is now current.
  • When the client posts transactions to the server, it also saves them as "online" transactions
  • Synchronizing transactions sent by the server are also saved as "online" transactions.
When the client is offline:

The client:
  • Loads the cached view
  • Gets both online and offline transactions
  • Synchronizes the view with the transactions
  • Posts the (offline) transactions locally
Offline transactions are posted using Sqlite:
public void SaveTransactions(PostTransactionsCommand ptc, bool isOffline)
{
  // Build the comma delimited PK list.

  StringBuilder csPkList = BuildCsPkList(ptc);

  // Write out the transactions...

  using (DbCommand cmd = sqliteConn.CreateCommand())
  {
    // Write out the view and container for which these transactions 

    // are associated.

    int id = WriteTransactionInfo(cmd, ptc, csPkList, isOffline ? 1 : 0);

    // For each transaction in the log associated with the view and container...

    foreach (DataRow row in ptc.Transactions.Rows)
    {
      // Write the transaction record.

      using (DbCommand cmd2=sqliteConn.CreateCommand())
      {
        int recId = WriteTransactionRecord(cmd2, row, id);
 
        // Each transaction record has one or more PK values that uniquely 

        // identify the row being operated on in the view's table.

        foreach (string pkcol in ptc.PKColumnNames)
        {
          using (DbCommand cmd3 = sqliteConn.CreateCommand())
          {
            WriteTransactionRecordPrimaryKeys(cmd3, recId, pkcol, row);
          }
        }
      }
    }
  }
}
The transactions directly correspond to the information that is managed by the DataTable Transaction Logger. For each transaction set, this consists of:
  • The view name
  • The primary key column names
In the code, you'll note that the transaction set is qualified not just by a view name but also by a container name, as the container concept is used to manage views that my be filtered in different ways.
For each transaction in the set:
  • The transaction type (update, insert, delete)
  • The column name being affected (not used for insert or delete)
  • The value type (not used for insert or delete)
  • The new value (not used for insert or delete)
  • The PK values that uniquely identify the record (used for all transactions)

How should transactional data be synchronized with the server when the application goes from offline to online?

I feel there's actually two parts to this question--how and when.

How?

The how is already addressed in the process described above for loading a DataView--the offline transactions are sent up to the server, the client gets an updated snapshot, and the local transactions are deleted.

When?

When is a much more interesting question. For example, for my client, the application is running 24/7/365 and the computer is contained within an enclosure. Rebooting or restarting the application is not desirable, so the client application needs to reconnect and re-sync automatically. The simpler case would of course be, just re-sync when the client logs in to the server. This is not a feasible scenario for my client. On the other hand, if it is feasible for you, then you can ignore all the issues with reconnecting while running.
When the ReconnectedToServer event is raised, the client goes through the following motions:
  1. Sets the client into a connected state
  2. Logs in
  3. Reloads the active views
The act of reloading the active views synchronizes the server and updates the client's view snapshot. The following code illustrates this process:
void OnReconnectedToServer(object sender, EventArgs e)
{
  // Block all command/responses until we're done here. 

  // Wait until a current command/response

  // is completed before entering here.

  lock (commLock)
  {
    // Raise the reconnecting event.

    RaiseReconnecting();
    // Restart the reader thread.

    connectedServerComm.InitializeReader();
    // Set the client to connected state.

    SetConnectedState();
    // Login.

    Login(username, password);

    // Walk through the active containers and sync the views 

    // in those containers.

    foreach (Container container in containers.Values)
    {
      // Get each view...

      foreach (ViewInfo vi in container.Views)
      {
        DataView dvNew;

        if (vi.CreateOnly)
        {
          // If it's a create only view (no data is loaded), create the view,

          // which synchronizes the

          // server with any transactions that occurred offline.

          dvNew = CreateViewIntoContainer(vi.ViewName, vi.KeepSynchronized, 
              vi.Where, vi.ContainerId);
        }
        else
        {
          // If it's a create and load view, load the view, 

          // which synchronizes the server with any

          // transactions that occurred offline and updates the local 

          // cache with the new server snapshot.

          dvNew = LoadViewIntoContainer(vi.ViewName, vi.Where, vi.OrderBy, 
              vi.DefColValues, vi.Parms, vi.ContainerId, 
              vi.KeepSynchronized, vi.IsCached);
        }

        // Reload the view data. This updates the existing view records, 

        // causing any processes

        // that were updating view records to have invalid rows. 

        // Therefore, such processes need

        // to synchronize with the commLock object. 

        // TODO: See ReloadView.

        ReloadView(vi.View, dvNew);
      }
    }

    // Raise the ReconnectFinished event.

    RaiseReconnectFinished();
  }
}
The ReloadView method is a brute force approach to synchronizing the in-memory DataView with the view that was received from the server. It really is awful, actually, but it does get the job done for certain requirements. It uses the ExtendedProperties feature of the DataTable class to block transaction logger events and then copies, row by row, field by field, the new DataView into the existing DataView.
protected void ReloadView(DataView destView, DataView newView)
{
  // Stop events, etc.

  destView.Table.BeginLoadData();
  // Stop the transaction logger.

  destView.Table.ExtendedProperties["BlockEvents"]=true;
  // Clear the entire table of all records.

  destView.Table.Rows.Clear();

  // Get each new row.

  foreach (DataRow dr in newView.Table.Rows)
  {
    // Create a row in the new data view.

    DataRow newRow = destView.Table.NewRow();

    // Copy the column values.

    foreach (DataColumn dc in destView.Table.Columns)
    {
      newRow[dc] = dr[dc.ColumnName];
    }

    // Add the row.

    destView.Table.Rows.Add(newRow);
  }

  // Accept all changes.

  destView.Table.AcceptChanges();
  // Re-enable events, etc.

  destView.Table.EndLoadData();
  // Re-enable transaction logging.

  destView.Table.ExtendedProperties["BlockEvents"] = false;
}
There are several issues with the reconnect process that I discuss in the "Issues" section below. However, one point here--the above code is not how to update the DataView in a production environment. Instead, the DataView should be synchronized using the existing DataRow instances. Care must be taken when dealing with rows currently being edited (for example, in-grid edits), and editing rows that are now deleted. The implementation of these mechanisms would itself be worthy of a separate article.

Question Time

Why Not Use Sqlite For The DataView Cache?
That's a very good question, and it was one that I struggled with for some time. There certainly isn't a right answer.
I decided that I wanted to keep a clear separation between the data view snapshot and the corresponding transactions. The client doesn't have any of the logic that the server does with regards to updating tables, and I didn't want to get to a place where I would even consider implementing the server side logic to update the client data view in a client-side database. I would, after all, still need to maintain the transactions separately so that they could be sent up to the server.
Another reason is that it's simpler. Rather than creating and managing the necessary tables within Sqlite, it's easier (and faster, in my experience) to serialize the data view to a discrete file.
And finally, the crux of the matter was the issue of the schema. While the schema is available to the client, as far as the client is concerned, the schema is there to help create empty views and access view properties such as regex validation, which is defined in the server's schema. The schema though is actually a somewhat dynamic thing. In many cases, I can update the schema without bringing down the server. This makes it very convenient to add new functionality to the enterprise. If I stored the view snapshot in Sqlite as a table, the client would have to also determine whether the scheme changed, delete out the old table and create the new one. At the moment, this seems like unnecessary complexity.
Why Not Use XML For Transactions Instead Of Sqlite?
Another good question. Again, there isn't a right answer. I chose Sqlite though with the hopes that it would be less verbose than XML and faster. However, the crux of the matter was that Sqlite provides built in encryption. If I stored transactions in XML, I would have to provide the encryption services. Unlike the data view cache, which is a snapshot and therefore is a one-time encrypt/decrypt process, transactions are always being added, involve two nested relationships, and need to be deleted when the server is synchronized. A database seems like a more natural persistence mechanism than a flat XML file, and not having to deal with encryption made Sqlite the more logical choice.

Issues

There are several issues with an offline client that must be addressed by any concrete implementation. These issues are not addressed in this article.

Authenticating The User

Normally the server authenticates the client. When the client is offline, the client itself needs to perform the authentication. And of course, user authentication is entangled with user roles and permissions.

Roles And Permissions

A simple solution would be to allow only the minimal roles and permissions when offline. More complicated solutions involve caching the role and permission tables and implementing the same server-side logic on the client. Yet again though, what happens if the administrator revokes a role or permission, but the user continues to have access because they are offline? How are transactions handled at the server that now should be disallowed because the role/permissions have changed? I feel that these are questions that cannot be answered generically and expect the solution to fit everyone's requirements. On the other hand, it should be possible to abstract the problem sufficiently to allow the application to specify the particular paradigm it wants to use, and of course provide a mechanism to extend that paradigm for requirements that are truly outside the box (or not considered initially).
Another mechanism might involve specifying which views can be cached and which views, under no circumstances, are ever cached. Program features could be disabled as determined by the availability of views that the feature requires. This is an option but again is specific to the individual application requirements and can only be supported abstractly.

Synchronization

Unless explicitly implemented by the client during startup, the current architecture does not synchronize the server with a data view's offline changes until that data view is actually loaded. This is relegated to the client implementation.

Master-Detail Synchronization

Synchronizing a detail view requires synchronizing the master views first. Or, more generally, any foreign keys in a view are a clue that there is a parent view that might need to be synchronized first. This is currently handled by the load order of the views, which is definitely not the ideal situation. These are issues that this architecture does not address and is relegated to the client implementation.

Dirty Data

When synchronizing offline transactions (and even during online operation), it's possible to end up updating a record that has been deleted or updating a value that has been changed by another client. These are issues that this architecture does not address and are relegated to the specific client/server implementation.

Server-Side Qualifiers

To reduce the amount of data being sent from the server to the client, and to make queries more efficient, we often resort to using serer-side qualifiers (SQL "where" clauses) to filter the view at the server. In an offline client, where the view is cached, server-side qualifiers that filter on client-side dynamic data will fail when working with a cached view. Examples include:
  • The user name or user ID to determine permissions
  • UI data that is used to qualify a load view command
These scenarios (and others) add a great deal of complexity to working with an offline client. The application requirements have to weigh the issues of data size, performance, and available offline features, while the developer has to also be very conscious of how they are interacting with the server and how that might result in preventing an offline client from actually working (or, even worse, give an offline client permissions that they would not normally have).

Reconnecting

As I mentioned previously, some of these issues are because of the requirement to re-establish the connection with the server without having to restart the client application. If that's not a requirement for your application, then life becomes a lot easier. That said, here's some considerations for automatic re-connecting while the application is running.

What happens if the server goes down during the reconnect process?

The critical issue here is, did the server get the transaction and can the transaction now be deleted from the client's transaction cache? Secondarily, how is the client transitioned back to a disconnected state smoothly? Ensuring that the local DataView cache is not corrupted is also critical.

What happens if the user is editing a record when a reconnect occurs?

Using the Row Transaction Sandbox (RTS), the user can be fairly well isolated from the view update process--while they are working in the sandbox, the view can be reloaded without the user losing their edits. Because the RTS itself uses a transaction logger, committing the changes is not affected by the fact that, in the above implementation, the concrete DataRow is now a new instance. However, if the row is now deleted, or the user in doing in-grid edits, the RTS does not come to the rescue. Again, the code presented above for synchronizing the DataView is a simple hack to get a prototype working.

How are inactive views updated, and when?

In addition to updating views that are currently loaded and possibly being displayed, the issue remains as to how inactive views are synchronized. Should this be a background task? Should one not even bother until the view is needed? Where is the balance in keep the client as synchronized as possible, dealing with bandwidth considerations (do you sync a client if they are on a slow connection?) and again other unpredictable application specific issues and requirements.

The Code

The download, as I've mentioned, is really a toolkit to explore one way of implementing a disconnected client and as a basic for considering more complex issues. You will note in the download that the server command processing implementation is completely missing (though the TcpServer code is included). You're essentially on your own for creating the server command processing. I felt that going into the server implementation would detract from this article, which is focusing on just the client side. I imagine there will be some people that grumble about this and the fact that I'm not providing a complete demo. If encouraged, I may write a follow up article for the minimal server implementation.
The code consists of the following folders, which I'll describe here:

Api

This consists of the core Api methods that an application would interface with to communicate with the server. The API architecture emphasizes that the application use a single interface for communicating with the server, and the API itself uses a single entry point for issuing commands and obtaining responses. The API also handles the connect/disconnect events (among others).

Cache

Has a simple static class that implements file-based caching.

Client

Provides basic and stub implementations that an application client would need to enhance. A simple template for synchronizing the client with server notifications is provided, as this is illustrative of the transaction logger and synchronization manager.

Comm

Implements the template connected and disconnected communication classes. This includes a complete reader thread and packet reader for the connected communication class. The disconnected communication class implements the transaction logging to Sqlite, a thread for attempting reconnects, and simulating message responses by working with the cached view data.

Crypto

Implements a wrapper for common encryption algorithms. This is a thin wrapper for constructing different encryption/decryption algorithms.

Logger

Includes the files for the transaction logger and synchronization manager. Please refer to the links listed at the beginning of this article for further documentation. The code here is however the most current code.

Misc

Miscellaneous classes--a data converter (similar to Convert.ChangeType), the KeyedList class, Stephen Toub's managed thread pool class, my ProcessQueue class, and a string helper class.

Packets

These are the basic command and response packets--login, load view, create view, post transactions, and sync view.

RawSerializer

The raw serializer code, as per this article.

TcpLib

The communications services classes, covered here and here.

Conclusion

This article is not the typical "here's a canned solution to a problem" article. Instead, I have attempted to discuss the issues surrounding an offline client architecture, the design decisions that I made within the context of other articles that I have written, and I have tried to identify issues that I felt were outside of the scope of a generic implementation and must be dealt with according to your specific needs. As I mentioned in the introduction, the code is not a turnkey solution but more like a kit that hopefully gives you some useful pieces to play with.


Download source code - 112 Kb

0 comments:

Reusable multithreaded tcp client and server classes with example project in VB.NET

7:01:00 am 2 Comments

  • A multithreaded server class that accepts multiple connections from a provided client class. Each client can send and receive files and text (byte data) simultaneously along 250 available channels.
ExamplePic.jpg

Introduction 

This example project contains three classes - TcpCommServer, TcpCommClient and CpuMonitor. With these classes, you will not only be able to instantly add TCP/IP functionality to your VB.NET application, but also most of the bells and whistles we're all looking for. With these classes, you will be able to connect multiple clients to the server on the same port. You will be able to easily: throttle bandwidth to the clients, and send and receive files and data (text?) along 250 provided channels simultaneously on a single connection.

Background 


When I first started looking for information or example code for tcp/ip communications in vb.net, I have to admit I was looking for it all. I was willing to accept the bits I needed to get started, but I was hoping, like you probably are, that I would find it all... some easily transportable classes that would allow me to do what I think most of us want when we first start looking into this - send and receive files, and our own data or text, all over a single connection, simultaneously. It's what we'll all need eventually, for one project or another when the time comes to build a client/server application. We'll also need to control the bandwidth at the server, or someone will use it all up downloading a large file... And oh - it would be nice if we could connect to the server on the same port from multiple clients, right?
I looked for a long time and never found anything right out of the box that would do what I need, so I decided to build it myself. Along the way I began to understand why people don't just freely give this kind of work away. It took me quite a bit of time coding, troubleshooting and testing this - but it is, after all, just a tool. It's what we do with it that will be worth something in the end.
So here it is.

Using the code 

First of all, both classes use delegates to do callbacks. All you have to do is pass the AddressOf YourCallbackSub while instantiating these classes, and then call start or connect Ie:
Dim _Server As New tcpCommServer(AddressOf YourCallbackSub)
_Server.Start(60000) 
Or:
Dim _Client As New tcpCommClient(AddressOf YourCallbackSub)
_Client.Connect("192.168.1.1", 60000) 
YourCallbackSub has to have the right signature  - you can see the right way to do it in the example. You can also specify the max bps while instantiating the server class. The default value is 9Mbps.
Throttling works along a 4k boundary if you set the bandwidth to be less then 1 meg, so you can use the traditionally expected bandwidth limits - ie: 64k, 96k, 128k, 256k, ect. After one meg, it's along a 5k boundary - so you can set it to more intuitive values: 2.5 meg, 5 meg, 9 meg, ect. For those who are interested - the throttling code checks the bandwidth 4 times a second.
Throttling will be important for you because these classes will do as much work as you let them - as fast as they can. On my dev machine, I was able to achieve transfer speeds of almost 300 Mbps copying one large file from server to client. But the cost of this was the client and the server's background threads using 100% of their respective CPU core's resources - not a good scenario for a server. On my dev machine, 9Mbps produced almost no perceptible cpu usage. In a production environment, I'm sure you will want to set this even lower.
To get a file from the server, there is a GetFile() method. Simply supply it with the path of a file on the server (a path local to the server). To send a file to the server, there's a SendFile() method. Both GetFile and SendFile are  handled by the classes without any other code needed by you - but if you'd like to track the progress if the incoming or outgoing file, you can poll the GetPercentOfFileReceived and GetPercentOfFileSent methods.

Channels 

We want to be able to do everything... right? And all over a single connection. You want to be able to send, say, a stream of screen shots, or video, or text, and not have to code your own way of separating one from another. The channel was my answer. every time you use the SendBytes method, you send that data with a channel identifier. When it comes out the other end in the callback, it arrives with the channel identifier you sent it with so you can tell your screenshot bytes from your text - if that's what you're sending. As I said above, you have 250 channels at your disposal (1 to 251).

The CpuMonitor class

Included in this project, you'll find the CpuMonitor class. This class is used to monitor the cpu usage of a thread. When running the example project, you will see some reporting of cpu usage in the status strip. This is the usage of the client's background thread - the thread actually doing the work of communicating with the server.

Points of Interest 

One thing worth noting here was an issue I noticed while testing these classes. As you would expect, I tested sending larger and larger files, and more and more connected clients all sending or receiving large files. At one point, I had one client pulling a >3 gig file while two other clients were also doing largish file transfers, and my bandwidth seemed to drop to almost nothing after 2 gig had been sent (almost nothing being between 1 and 30 Mbps - I was testing without throttling to see this effect). After much testing, I realized that this is a hardware limitation on my machine. I think it has to do with the hard drive cache. I was able to mitigate this effect by inflating the filestream's buffers in the FileWriter object in the client for large file transfers to the client... but if you notice this effect on your system, this is what it is. Yet another reason to throttle bandwidth.




Download TcpCommExampleProject - 39.7 KB

2 comments:

Numbers to Indian (Hindi) Words Conversion In Unicode

6:57:00 am 0 Comments



NumbersToHindiWords.png 

Introduction


I recently, post an article to convert numbers to words and then I realize that the same logic can be used to convert any other language so here I am starting with Hindi (Indian) language but using Unicode to get the converted numbers in Native language. As you know, the benefit of Unicode, you don’t have to install the font and can display the Native language in web pages as well.
The entire conversion relies on these three arrays. Now, similar arrays can be developed for another languages and conversion can be made. Granted you have to understand how the numbering logic works for that language. Indonesian and Chinese are coming soon!

Private HundredHindiDigitArray() = _
    {"", "एक", "दो", "तीन", "चार", "पाँच", "छह", "सात", "आठ", "नौ", "दस", _
    "ग्यारह", "बारह", "तेरह", "चौदह", "पन्द्रह", "सोलह", "सत्रह", "अठारह", "उन्नीस", "बीस", _
    "इक्कीस", "बाईस", "तेईस", "चौबीस", "पच्चीस", "छब्बीस", "सत्ताईस", "अट्ठाईस", "उनतीस", "तीस", _
    "इकतीस", "बत्तीस", "तैंतीस", "चौंतीस", "पैंतीस", "छत्तीस", "सैंतीस", "अड़तीस", "उनतालीस", "चालीस", _
    "इकतालीस", "बयालीस", "तैंतालीस", "चौवालीस", "पैंतालीस", "छियालीस", "सैंतालीस", "अड़तालीस", "उनचास", "पचास", _
    "इक्यावन", "बावन", "तिरेपन", "चौवन", "पचपन", "छप्पन", "सत्तावन", "अट्ठावन", "उनसठ", "साठ", _
    "इकसठ", "बासठ", "तिरेसठ", "चौंसठ", "पैंसठ", "छियासठ", "सड़सठ", "अड़सठ", "उनहत्तर", "सत्तर", _
    "इकहत्तर", "बहत्तर", "तिहत्तर", "चौहत्तर", "पचहत्तर", "छिहत्तर", "सतहत्तर", "अठहत्तर", "उनासी", "अस्सी", _
    "इक्यासी", "बयासी", "तिरासी", "चौरासी", "पचासी", "छियासी", "सत्तासी", "अट्ठासी", "नवासी", "नब्बे", _
    "इक्यानबे", "बानबे", "तिरानबे", "चौरानबे", "पंचानबे", "छियानबे", "सत्तानबे", "अट्ठानबे", "निन्यानबे"}

  Private HigherDigitHindiNumberArray() = {"", "", "सौ", "हजार", "लाख", "करोड़", "अरब", "खरब", "नील"}
  Private HigherDigitSouthAsianStringArray() As String = {"", "", "Hundred", "Thousand", "Lakh", "Karod", _
                                                         "Arab", "Kharab", "Neel"}

  Private SouthAsianCodeArray() As String = {"1", "22", "3", "4", "42", "5", "52", "6", "62", "7", "72", _
                                             "8", "82", "9", "92"}
  Private EnglishCodeArray() As String = {"1", "22", "3"}

  Private SingleDigitStringArray() As String = {"", "One", "Two", "Three", "Four", "Five", "Six", "Seven", _
                                                "Eight", "Nine", "Ten"}
  Private DoubleDigitsStringArray() As String = {"", "Ten", "Twenty", "Thirty", "Forty", "Fifty", "Sixty", _
                                                 "Seventy", "Eighty", "Ninety"}
  Private TenthDigitStringArray() As String = {"Ten", "Eleven", "Tweleve", "Thirteen", "Fourteen", _
                                              "Fifteen", "Sixteen", "Seventeen", "Eighteen", "Nineteen"} 

Background


Hindi numbering system, like other South Asian numbering system, is very alike. The last three digits from right are ready in one way then then higher order digits are read similar to the 10th place digit but with a suffix of higher order digit word.

Example:
12,12,112 = Twelve lakh twelve thousand one hundred twelve
12,00,000 = Twelve lakh
      12,000 =                    twelve thousand
            112 =                                               one hundred twelve
12,12,112 = बारह लाख बारह हजार एक सौ बारह
12,00,000 =
बारह लाख
      12,000 =                       
बारह हजार
            112 =                                                
एक सौ बारह

Code Flow

The entire process is basically array and string manipulation. The primary goal is to find the correct index corresponding to the number and its position and then pulling the corresponding word out of the array shown above.

Below is the main function that converts giving number to Hindi words. Zero is exceptional case so we have to be careful at every step when working with digit zero. The very first thing we do is convert the given number to string and then to an array, by calling NumberToArray for example, 1234 to “1234” then to {1, 2, 3, 4}.

Now the fun begins. We first find out in which place the given digits falls in like, unit, tenth, hundredth, and so on by using SouthAsianCodeArray. The logic behind this array is very simple, explained later in the article.  Once we know the place of the digit we can trisect the case as if it’s in unit place, tenth place and other place. When working with these numbers, we take advantage of both backward (i variable) and forward (j variable) indices.

Private Function HindiStyle() As String
    Dim amountString As String = Amount.ToString

    If Amount = 0 Then Return "शून्य" 'Unique and exceptional case
    If amountString.Length > 15 Then Return "That's too long..."

    Dim amountArray() As Integer = NumberToArray(amountString)

    Dim j As Integer = 0
    Dim digit As Integer = 0
    Dim result As String = ""
    Dim separator As String = ""
    Dim higherDigitHindiString As String = ""
    Dim codeIndex As String = ""


    For i As Integer = amountArray.Length To 1 Step -1
      j = amountArray.Length - i
      digit = amountArray(j)

      codeIndex = SouthAsianCodeArray(i - 1)
      higherDigitHindiString = HigherDigitHindiNumberArray(CInt(codeIndex.Substring(0, 1)) - 1)


      If codeIndex = "1" Then 'Number [1, 9]
        result = result & separator & HundredHindiDigitArray(digit)

      ElseIf codeIndex.Length = 2 And digit <> 0 Then 'Number in tenth place and skip if digit is 0
        Dim suffixDigit As Integer = amountArray(j + 1)
        Dim wholeTenthPlaceDigit As Integer = digit * 10 + suffixDigit

        result = result & separator & HundredHindiDigitArray(wholeTenthPlaceDigit) & " " & _
                                       higherDigitHindiString
        i -= 1

      ElseIf digit <> 0 Then  'Standard Number like 100, 1000, 1000000 and skip if digit is 0
        result = result & separator & HundredHindiDigitArray(digit) & " " & higherDigitHindiString
      End If

      separator = " "
    Next

    Return RemoveSpaces(result)
End Function 

Remove extra spaces:

During the process a space or two get attached in between the words so for the cleanup I use the RegEx and call the RemoveSpaces function as:
Private Function RemoveSpaces(ByVal word As String) As String
    Dim regEx As New System.Text.RegularExpressions.Regex("  ")
    Return regEx.Replace(word, " ").Trim
End Function 

Number formatting (or grouping):

There is another public function FormatNumber which basically calls a private FormatNumberPerLanguage in the Converter class. This FormatNumberPerLanguage will format group based on the provided regional name which is “hi-IN” in this case. A simple use of CultureInfo class.
Private Function FormatNumberPerLanguage(ByVal culterInfoName As String)
    Dim ci As New System.Globalization.CultureInfo(culterInfoName)
    ci.NumberFormat.NumberDecimalDigits = 0
    Return Me.Amount.ToString("N", ci)
End Function

Points of Interest  

These arrays that helps to find the place where the numbers falls in are quite important and interesting. For example, in 123456 number, from right, 1 is at 6th position. Now from the AsianCodeArray the 6th item is "52" which tells two things:

a) the given number is in tenth position (of some order)
b)  and the higher order is in Lakh's position because of the first letter of 52 is 5 and 4th item (5-1 = 4) in HigherDigitSouthAsianStringArray or HigherDigitHindiNumberArray is Lakh or लाख

This is how I determine the higher order prefixing word!


Download NumbersToIndianWords.zip - 14.58 KB  

0 comments: