Tuesday, November 28, 2017

Questions about Machine Learning


If you haven't read the first in this series of articles on building enterprise Machine Learning applications with WebFOCUS and Python, you can find it here: Part I.

Before diving into Python details, let's consider some important questions: "What is Machine Learning" and "Why is it important?"


Traditional Software Programming 
Throughout most of our history of data processing, application developers have given computing machines explicit instructions on what to do and how to do it. These computers followed step-by-step commands without questioning the developers, a blind obedience that often resulted in "bugs" due to human error. For decades, we have "programmed" classic software applications by typing out commands on a keyboard.

Today is a turning point. With more robust computers with sophisticated capabilities, a developer can now provide a machine with pre-defined algorithms and training data (where the answers to the questions are already given). The machine can test the different algorithms to see which provides the most right answers. Using this as a model, the machine can then move on to new data that doesn't already have the answers given and make predictions.

Supervised and Unsupervised Machine Learning
A classic example comes from a 1936 paper describing fifty observations of three types of Iris flowers. A gentleman named Edgar Anderson spent some time measuring and documenting the lengths and widths of both the petals and sepals of 150 different Iris flowers evenly grouped into three different species. He labeled these observations as being from an Iris setosa, Iris virginica, or Iris versicolor.
Frank Mayfield photo: Iris virginica shrevei BLUE FLAG

After analyzing Anderson's Iris details, a statistician named Ronald Fisher wrote a paper on how we could distinguish these three Iris species based on the relationships between petal and sepal measurements. In other words, you could classify the flower without being an expert who could make a visual identification. In 1936, Fisher used a pencil and paper to compute this classification work but today we can use Python or R for what would be considered "supervised machine learning."


We have our choice of a variety of techniques for supervised machine learning, including: classifications, regression (predict a real value), decision trees (which can be used for classifications and regression), and dimension reduction (figuring out the important parameters and ignoring the others). You see these types of machine learning in a range of industry uses such as: credit ratings, spam detection, predicted values, sales prospecting, employee churn, and so forth.

A well-known example of supervised machine learning comes from Target Stores leveraging their baby registers to build a model of the buying habits of pregnant women. After building a predictive model from that labeled training data, Target software could search through credit card purchases to predict which Redcard holders might be pregnant and proactively reach out to them with recommendations for diapers, baby clothing, formula, and so forth. The shocker here is that Target could send coupons to a lady prior to her announcing the pregnancy to her family.

Our next step in the evolution of machine learning would naturally go from humans "supervising" machines and providing them with training material to letting them go unsupervised with unlabeled data. Let's see what machines can do without our assistance. The machine is given freedom to look for hidden structures, patterns, and relationships within the data. We humans don't really know what the outcome will be but have our fingers crossed it will be useful (perhaps profitable, beneficial to society, or at least insightful).

In unsupervised machine learning, the computer is not able to gauge the accuracy of its predictions. This still takes a human to evaluate the results. Some solutions might not even be clear-cut between supervised and unsupervised and fall into a category of "semi-supervised" machine learning where some but not all data labels are provided.

Reinforced and Deep Learning 
A further advancement in machine learning would be "reinforcement" learning. This borrows some theories from human psychology and tries to get machines to understand the concept of a "reward," providing incentives to maximize that goal. To do this, the machine may need to break down the problem into many sub-problems, store each solution, and later reuse those answers as part of larger problems (considered "dynamic programming" techniques or "dynamic optimization"). This is still unsupervised learning, so the machine doesn't know right or wrong but instead focuses on its performance, balancing exploring new information versus maximizing what it already knows.

Since we are trying to make the machine behave like a human motivated by rewards, we might as well just go all-in and try to simulate the human brain. Here we are in the realm of "deep" or "hierarchical" learning where machines work less with task-specific algorithms and rely more on interpreting and transmitting information like we would see in brain neurons. Use cases of deep learning are in the news: machine vision (e.g., face recognitions, classifications of animals, and self-driving cars not running into people or things), speech recognition (think of your favorite voice-activated personal agent: Siri, Cortana, Alexa, Google, etc.), and natural language processing (e.g., chatbots).

Many experts believe that those individuals or companies who tackle machine learning first will go on to become extremely rich, famous, and perhaps powerful.

You can also find many people giving dire warnings about the progress being made so that machines can learn like humans. Learning is one thing; machines are not necessarily yet acting currently on their decisions but this could change in the near future. As we become comfortable with machines making decisions for use and acting on our behalf (e.g., paying your bills, choosing the best flight and hotel for your trip, ordering food for your fridge, making stock trades in your name, or modifying your retirement portfolio), we are sure to give them more freedom. With some imagination, we have to wonder what if machines determine the best solution to a problem is to eliminate us humans and then they follow up with an action plan?

Next
For now, we can leave that topic to futurists and sci-fi authors. We'll ignore the development of Skynet and instead look at how Python implements machine learning features.

Wednesday, November 22, 2017

Enterprise Machine Learning Apps with WebFOCUS and Python


WebFOCUS integrated with Python for Enterprise Machine Learning App Dev

Integrating Powerful Technologies

A while back, I showed how WebFOCUS could be integrated with the R statistical language and mentioned that the same was true for Python; I just never fully showed how that would work. Now is a good time to do that. 

If you missed the original posts and are interested in how to call R from within Information Builders' enterprise BI product WebFOCUS, please follow these links (otherwise, keep reading for Python):

Three Complementary Tools

The WebFOCUS, R, and Python languages all have something in common: each is interpreted at run-time. This gives the savvy app developer like you the ability to dynamically generate code for each language based on real-time criteria such as user input. I have built completely data-driven systems where the user's selections caused a unique procedure to be automatically created and executed. These three interpreted languages fit together nicely for dynamic applications.

Why?

But why not just one or the other? Why bother integrating multiple languages? 

WebFOCUS is a powerful analytics product that can virtually do anything you want for secure, enterprise business intelligence. You can implement WebFOCUS on most platforms, can scale it up and out to millions of users, can use it to read almost any enterprise data source, and get high-quality analytics for practically any viewing device (for more of my opinions, read this article).

But WebFOCUS does not necessarily have built-in features for the emerging artificial intelligence applications.

While R and Python can be extended via powerful statistical and machine learning libraries, neither can match the enterprise data reach and user interface capabilities of WebFOCUS.

Therefore, we can slide R and/or Python into the middle of a WebFOCUS procedure for a truly enterprise machine learning application. R and Python are both open-source software (your organization is probably already using them), so adding machine learning to WebFOCUS can be done for a small price.

Next on Deck:

For now, I'm stopping here with this general overview of WHY you might want to integrate these powerful analytics tools. In the next part, I'll get closer to the HOW but will first take a look at Python and its capabilities for modern machine learning. Click here to continue reading.

Question:

Can you think of a place within your organization for enterprise machine learning apps, merging the power of a commercial enterprise BI product with the AI capabilities of open-source software?

Wednesday, November 15, 2017

Kick-start your WebFOCUS enterprise BI projects with expert mentoring

Two Weeks to Better BI!

Engage me for two weeks to shift your BI team into a higher gear. During the first week, I will provide a holistic view of the WebFOCUS enterprise BI product, how the various components work together, details on using the tools, and a proven approach to doing BI application development. In addition to all of the important core topics, I can cover unique issues impacting your organization. In the second week, I will work side-by-side with your BI team to build features of a custom WebFOCUS application.

Qualified WebFOCUS Expert

Having worked with many of the leading WebFOCUS customers, I can demonstrate how they were successful with the product.

I have worked with Information Builders, the software vendor of WebFOCUS, for decades: as an IB customer working in corporate IT; in IB Professional Services (instructor, consultant, and regional manager); in IB Technical Services (regional technical manager); and as a partner.





During these many years, Information Builders have awarded me multiple certifications, including:
  • WebFOCUS Business Analyst Certified Associate
  • WebFOCUS Application Developer Certified Associate
  • WebFOCUS Application Developer Certified Professional 
  • Certified instructor of variety of 4GL language topics
  • Certified instructor of product internals (reporting, maintenance, and database)
  • Certified instructor of advanced database design

Mentoring Topics 

We can customize the actual topics to fit your needs, but I typically cover areas such as:
  • Architecture, history of the product, underlying languages
  • Integration of the product with other technologies (e.g., R, Python, and SQL)
  • Agile project methodology for approaching BI applications

I dive into the WebFOCUS product details such as:
  • Data adapters, metadata (synonyms, Business Views, etc.)
  • Procedures, symbolic variables, scripting (Dialogue Manager)
  • Launch pages, controls, scripting (JavaScript)
  • BI Portal, responsive design, mobile
  • Security (built-in, custom, application-specific)
  • Schedules, distributions, alerts
  • Debugging, handling errors/no-data scenarios, performance optimization

I will cover the WebFOCUS tools such as:
  • BI Portal Designer
  • App Studio (for development of HTML, procedures, metadata, etc.)
  • ReportCaster scheduling and distribution
  • InfoAssist (end-user report writer)
  • Administrative consoles

We will then put all of this learning to good use by starting the development of a custom WebFOCUS application useful to your organization.

Contact Me

Contact me now to schedule an onsite visit: DLautzenheiser@Kencura.com.

Friday, October 20, 2017

Why I Choose WebFOCUS for BI/Analytics


WebFOCUS, the enterprise BI/Analytics platform from Manhattan-based Information Builders, has been around almost as long as the web browser. It's the product of choice not only for many large, global firms, but also for many small-to-medium sized companies.

It's also my BI tool of choice. Here are a few reasons why I'm confident recommending WebFOCUS.

Powerful yet Easy

WebFOCUS provides features for all BI users, both internal and external to the organization. General business users can have intelligence automatically delivered to them or users can open intuitive "InfoApps" on their desktop, tablet, or smartphone. Power business users can build their own sophisticated BI applications using the InfoAssist tool with browser-based GUI drag-and-drop features and share them with others in the "BI Portal." Hardcore technical developers can build complex enterprise BI applications using the Windows IDE called "App Studio."

Scalable

WebFOCUS can scale up, able to run on a variety of platforms based on your current needs. Should your usage volume change, WebFOCUS applications are portable and can be moved up from distributed environments to large mainframes. WebFOCUS can scale out, where servers are organized into load-balanced clusters. Whether you have a small group of twenty users or a global audience of twenty million, WebFOCUS can handle the volume.

Any Data Source

Not only can WebFOCUS run on any platform, it has adapters to read practically any data source. Actually, the sources are not even limited to what you would consider "data." Sources can also be applications (e.g., JD Edwards and SAP) and web services (which might be wrappers for programs written in legacy technologies).

Sitting on top of the data source is a powerful metadata layer that can contain items such as: user-friendly business names, help comments, calculations (mathematical expressions, string manipulation, IF/THEN/ELSE logic, function calls, SQL expressions, etc.), security rules, relational joins, hierarchy/dimensional grouping, and other important data enhancements.

All WebFOCUS tools use the same metadata layer, giving your organization a single version of the truth in all generated content.

Highly Dynamic, Multi-Step Procedures

The WebFOCUS engine interprets users' requests at runtime. A powerful scripting and symbolic substitution feature allows a single procedure to be dynamically changed into tens of thousands different permutations based on input selections and logical conditions. When I do conversions of other technologies, it is not unusual to convert hundreds of related legacy reports (e.g., those written in SAP BO Crystal Reports or WebI) into one single WebFOCUS procedure.

Unlike many BI products, these procedures are not limited to a single data query and formatting step. Instead, WebFOCUS can run sophisticated multi-step processes that branch differently based on user input or specific conditions (such as the day of the week, or whether certain data exist, or whether an event has happened or not).

A WebFOCUS procedure is not limited to just your typical reporting and graphing. Instead, the product provides a full application development language capable of building complex data systems. These BI applications can be extended with other technologies, such as C/C++, Java, R, and Python.

High-Quality Corporate Support and Product Enhancements

It's actually quite hard keeping up with all of the enhancements going into WebFOCUS. Being a privately-held company, Information Builders isn't forced to do things based on quarterly share prices. Instead, they are proud of being able to listen to their customers and coordinate product direction based on what they hear. Their customer support group is world-class and continually winning industry awards.

I'm a Fan

I've seen dozens of successful WebFOCUS enterprise BI implementations and proud to have been personally involved with many of them. If you would like to talk about WebFOCUS, feel free to reach out to me.

Sunday, June 4, 2017

IB Summit 2017 in Grapevine, Texas



Several people have reached out to see if I am attending the IB Summit 2017 the week of June 5th in Grapevine, Texas. Due to projects, I cannot make it this year but perhaps I'll see you there next time. 

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.