Posts

Does AI Understand Anything? Yes, It Does!

Image
(Read time ~2 mins) Before we contemplate, attempt, or contradict this assertion, let's ask ourselves a few more questions. Answering these will lead us to a satisfactory answer to the core question: Does AI understand? Can AI learn? There should be no doubt about this by now. After all, learning—or training your models—is what makes AI possible. Call it learning or call it training, it does not matter. If you still have doubts, consider how an infant learns to speak. It does not understand the rules of grammar, nor does it grasp the meaning of most words it hears adults speak. Yet, simply by listening over and over again and connecting those sounds with its state of mind (context), it learns words, phrases, and eventually babbles. It learns to read and write much later, but long before that, it starts using spoken language successfully. Our AI models do the same. By subjecting them to gigabytes of training data and examples, they learn and apply concepts successfully. They improve...
Post a Comment
Read more

When Should You Spin Up Your Next Apache Kafka Cluster?

  This post is published on the Confluent blog:  https://www.confluent.io/blog/kafka-clusters-and-cluster-management/
Post a Comment
Read more

Apache Kafka with Control and Data Planes

This post is published on the Confluent blog: https://www.confluent.io/blog/apache-kafka-security-and-governance-with-control-and-data-planes/
Post a Comment
Read more

Agile & Architecture

With this post I am trying to address the biggest myth about Agile, "We are agile, we don't need architecture or design". Let me start by supporting the myth first. For a  product that has been in the market for 10 years or more, this is not all myth but reality. For a product that has existed for such as long time, there is probably no need to do any design or architecture anymore. It was probably designed, when it was first conceived or when it underwent major upgrade. It may not have been formally documented but its creator must have had a white boarding session and picture on the whiteboard. It may have been formally documented as part of the product documentation for developers. However as it matured and evolved to a certain stage, it stopped changing much. The only change it undergoes nowadays is cosmetic or for superficial features. Under this scenario I would agree to a statement I heard from a supervisor, a self-proclaimed agile champ, when somebody in the team ...
Post a Comment
Read more

A framework for generating Payment(SWIFT) messages

Image
It was late 2007, world was on the brink of credit crisis. I was a lead architect on a project for a large American bank on a consulting assignment. The project required generating and consuming SWIFT messages and I was thinking hard on how to make it easy for my team to exchange SWIFT messages as part of the system that was being developed. After searching around I could not find a good software or a framework that would do the job and so I set out to design one myself. I soon realized that composing SWIFT messages element by element was going to be hard as it required the developers to understand the rules and grammar of SWIFT. If I were to ask my developers to learn SWIFT and then work on the module of the system that would be responsible for exchanging SWIFT messages, I know I would be on a suicidal mission. Most of the developers in my team were novices, let alone understanding SWIFT they didn't even know enough programming. You know how the offshoring companies work, its all...
Post a Comment
Read more

A Brief Summary of SOA Patterns

In this post I am attempting to summarize the SOA patterns that were published in the book, "SOA Patterns" by Amon Rotem-Gal-Oz. This is based completely on my understanding of the matter presented in the book and there is a possibility of a gap in my understanding, if so, please feel free to correct me by leaving a comment. Please also note that this is not the only book on SOA patterns but there is another one, "SOA Design Patterns" by Thomas Erl. The reason I decided to read the book by Amon Rotem-Gal-Oz is because it has a foreword from one of my favorite authors, Gregor Hohpe, one of the author of Enterprise Integration Patterns. With this post let me discuss the patterns presented by Rotem-Gal-Oz. For the patterns from Thomas Erl I may write another post. The very reason I am writing this post is to make life easier for those who don't have time to read  through 300 page book on SOA patterns but want to have a very high level & quick understanding o...
Post a Comment
Read more

How we came across the need for Big Data way back in Y2K

Image
We were working on a batch job and the requirement was to finish processing all the transaction in a database within a maintenance window of less than a hour. The requirement was to read hundreds of thousands of records, call a service and validate and enrich those records using reference data returned by the service (Back then we had no web services, we had CORBA or IIOP (Java RMI/IIOP) to invoke those services). Now the challenge was the time window that was very brief. We figured out that reading the records sequentially was not an option and we had to process those records in parallel. So I proposed we could use batch jobs running in multiple processes spread across multiple nodes to read and process the records from the database, the slaves or Yes, the "Mappers"!  as in MapReduce. Now how could we prevent the parallel batch jobs from stepping on each others toes and avoid picking up the same set of records for processing. How could we avoid this duplication? use row l...
Post a Comment
Read more