C# .NET client to publish 5 million events to Universal Messaging channel\IS Publishable document

In this blog article I have created a C# .NET client using Universal Messaging C# .NET API. This is in contrast to my previous two articles where I used Java and Message builders(as described in IBM official documentation) and Dynamic Builders to compose a Protobuf event and publish large amount of data to UM Channel\IS Publishable document. I took on the effort to do the same thing in C# to see if there will be a performance difference or if there is a different way to send messages to UM Channel more efficiently and to my surprise the performance metrics are so different between both implementations. I have described more in detail about the performance differences with same UM and IS setup in detail in the blog. Generate the C# code from the proto file. Below command is for running it from the same folder where the proto file is located. protoc --csharp_out=. Sales_pdt.proto

This will generate a new cs file called SalesPdt.cs as show below.

At this point we have the auto generated csharp file generaed by the protoc compiler that we can import in to our existing solution and generate a nProtobufevent from the data and then loop over each generated event and publish to UM.

I have listed out a step-by-step instructions on how to implement a C# .NET program to generate a proto buf event and publish the event to UM. You can download the pdf document here.

For any questions or inquireies feel free to reach out to aarremreddy@silverspringtech.com

Program.cs

using com.pcbsys.nirvana.client; using Google.Protobuf.Reflection; using Google.Protobuf; using System.Text; using System.Reflection; using MyChannels.Nirvana; using IBM_MQ; nSession mySession = UMUtilities.UMConnect("nsp://localhost:9000"); nChannel myChannel = UMUtilities.setChannel("wm/is/test/Akshith/docs/SalesDataDoc", mySession); String[] records=File.ReadAllLines("C:\\Users\\test\\Downloads\\5m-Sales-Records\\5m Sales Records.csv"); Console.WriteLine(DateTime.Now+" -> Start publishing documents to UM."); for(int i=0; i<<records.Length; i++){ myChannel.publish(Sales.generateSalesEvents(records[i])); } mySession.close(); Console.WriteLine(DateTime.Now + " -> Finished publishing documents to UM."); Console.ReadLine();

Sales.cs

using com.pcbsys.nirvana.client; using Google.Protobuf; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace IBM_MQ { class Sales { public static nProtobufEvent generateSalesEvents(String inRecord) { String[] fields = inRecord.Split(','); dli_test_Akshith_docs_SalesDataDoc outDoc=new dli_test_Akshith_docs_SalesDataDoc(); dli_test_Akshith_docs_SalesDataDoc_salesData salesData = new dli_test_Akshith_docs_SalesDataDoc_salesData(); salesData.Region = fields[0]; salesData.Country = fields[1]; salesData.ItemType = fields[2]; salesData.SalesChannel = fields[3]; salesData.OrderPriority = fields[4]; salesData.OrderDate = fields[5]; salesData.OrderId = fields[6]; salesData.ShipDate = fields[7]; salesData.UnitsSold = fields[8]; salesData.UnitPrice = fields[9]; salesData.UnitCost = fields[10]; salesData.TotalRevenue = fields[11]; salesData.TotalCost = fields[12]; salesData.TotalProfit = fields[13]; outDoc.SalesData = salesData; Sales_pub_publish_envelope env=new Sales_pub_publish_envelope(); env.TransactionId = "asddfad"; outDoc.Env = env; nProtobufEvent evt = new nProtobufEvent(outDoc.ToByteArray(),"salesData"); return evt; } } }