Abstract

IT organizations are under pressure to deliver more bang-for- the-buck. Intelligence delivered on existing corporate data is usually viewed as low hanging fruit that.  For small organizations, executing these projects cost-effectively remains a daunting challenge. Based on over 10 years of working small and medium sized companies, we share 10 tips that can help.

10 things you can do to get your BI initiatives done for less:

There is always a reason to do it better, faster and cheaper but the reasons to do more with shrinking budgets seem to be mounting up now more than ever. The financial crisis and the increasing global competition have led to greater demands to view their business differently and to react to changes faster. To add to this an ever changing plethora of new technology paradigms has cropped up promising cheaper and faster BI enablement. All this creates pressure on IT firms to deliver value from existing data assets quickly and cheaply.

In large organizations, BI projects will typically be an exercise lasting a couple of years, multiple geographies, multiple organizational units and a wide range of technologies and systems. Small and Medium sized companies, however, struggle to get executive support to view data as a centralized corporate asset and to clearly demonstrate value it can generate. Projects need to be executed faster and cheaper and need to be more aligned with business goals so as to encourage future investment in the initiatives. Visions of large, complicated and time consuming warehouses are a turn off for business sponsors. They will begin to question the value this insight will generate for their business. Through our experiences working with small and medium sector businesses we’ve listed down 10 things we felt were instrumental in demonstrating BI value to SMEs.

1.     Business Intelligence not Business Re-engineering

By its very nature, BI projects are cross-functional and cross-system. This will usually bring up questions on “how things are done” in the business.  While the ultimate goal of these projects remains improvement of overall business processes overall, these discussions should be tightly controlled and quickly resolved.  Without this, these discussions will take a life of their own while development teams wait for guidance or readjust to changing expectations. Senior management must provide clear guidelines on the charter of the BI program. Usually situations that go beyond a certain threshold of complexity or time should be deferred to a later phase or stage, when the ROI makes sense. One client we worked for set the threshold to 4 work-hours of discussions on a topic before they decided to re-allocated to a slower moving, later-phase discussion. That work-stream carefully debated the issues, analyzed the impact of business project changes and developed high level executive buy-in for the required business changes. Once a decision was reached (usually spanning months), the tangible work-items were brought back into the faster development work streams.

Breaking down your work items this way leads to efficient utilization of the resource pool and avoid the “analysis paralysis” that many BI projects fall into.

2.     Iterative project execution

SME BI initiatives have a characteristically small leash, that is, from the time the report is initiated to the point that value is noticed by business has to be a short cycle – typically no more than a couple of weeks. Executive sponsorship exists to the point of initiating projects – usually triggered by “everyone else having it”, “having seen it somewhere”. Few SME executives are sure about what the BI experience will be like and how significantly it will impact their business situation. Small iterative cycles help ensure that negative feedback can be quickly rectified and that positive feedback can be translated into goodwill that drives the overall BI initiative forward.

We recommend using an Agile based SLDC like SCRUM or XP. Specialized variants of these exist and are the norm in many consulting firms. These till typically involve

–          A small analysis cycle, not lasting more than a couple of days. Analysis will look at existing reports and determine the most commonly used features. Implementation then focuses on implementing the 20% features used 80% of the time.

–          Development cycles ranging 2-3 weeks focused on getting on feedback often. The objective is usually to add a small scope or improvement quickly and share it with the users.

–          Quick 1-2 day testing cycles that validate key data points.

–          Re-evaluation of current and future positions and detailing out the next sprint.

A couple of things to be weary of with agile project delivery:

1. Have a broader roadmap in place. It is easy to get lost in small iterative steps that lead you in circles. Keep a clear, long term, executive approved goal in mind and align your steps in that direction. Without a clear direction, the small iterations will involve huge amounts of refactoring and rework.
2. Start with your low-hanging value propositions and stop when your users stop perceiving value. If no one really uses a report, it doesn’t really exist.
3. Manage your resource pool very tightly. Agile delivery models are usually billed T&M and companies end up paying the price of delays and indecision. If you have a large scope of work that you are very sure of, consider a fixed price agreement with your consulting partners.

3.     Rely heavily on user-driven reporting

Report visualization will typically be around 10-30% of your solution cost. User driven reporting tools have promised user-enablement for the purposes of relational and analytical reporting for a while. Of late, these tools have become feasible for SMEs due to the low cost of ownership. These include CapEx friendly SAAS models and as free tools available in the open source community. Enabling users to generate their own reports (or variants of these reports) reduces the IT budget required for the implementation and results is a higher sense of control for the business users.

Users can view the same information in many different ways and formats. One organization we worked with had 910 variants of a sales report customized for each store in their business. They had spent over a year evolving and maintaining the various versions and by the time they reached us, were exhausted of the exercise. These reports were constructed a couple of years ago when user-driven reporting was not within the reach of the SME. Once the toolset was implemented, the users were presented with a data model driven framework that let the users create a personalized view of the model as a report. The users loved the idea and ended up customizing their views to fit their needs exactly. IT was involved in the preliminary training and mentoring of the users and focused on the higher level questions of the data-model and consistency.  Users also shared their views with peers with similar requirements enabling power-users to share their insight with the broader audience.

4.     Associate an internal functional expert

No matter how good your consultant claim to be, they don’t know your business as well as you do. Functional experts in the organization will usually take the shape of the lone IT champion who has been building reports and extraction packages in the years before the BI initiative. These people will usually have a lot of context including issues with data or the history of report’s evolutions – thing that are invaluable in understanding the current state and expectations of the system. In small organizations, knowledge of this level is rarely documented which makes it fairly cumbersome to explain to an external consultant.

This expertise, however, usually means that these people will usually be extremely involved in maintaining existing reporting assets and there is be resistance in adding them in a large new initiative. In our opinion, involvement of these experts pays off significantly. Key advantages are:

1. Reduced friction and frustration while gathering business requirements
2. Reduced ramp up time for the development and analysis teams. The analysis also becomes much more comprehensive since it includes the net sum of the expert’s experiences.
3. Reduced rework
4. Increased reuse of existing assets.

5.     Reuse existing collateral

Even if the SME is new to BI platforms, they have probably been reporting for a while. These usually take the form of canned reports from the ERP or user-spreadsheets driven off of external data or ERP data feeds. These spreadsheets are usually built and used by individuals for their own data analysis and reporting.

It seems obvious to assume that existing assets will be reused but there is a tendency to begin to consider these as obsolete once the “new” BI initiative begins. Existing reports should be viewed as assets to be built on top of (vs. instead of). Rather than attempting to phase them out, they should be used as baselines for the new reports.

Also, many BI platforms now a days are collaboration platforms and their value is derived not only from enabling new BI collateral by making existing collateral more accessible. The key benefits:

1. Everyone getting the same version of the truth. Versions of spreadsheets flying around in emails are a productivity killer and a common place to edit and review other people’s changes helps reduce the effort of merging and connecting data sources.
2. Enabling everyone to get the best version of a reports. One customer we worked for recently had 8 versions of a fulfillment report – one for each of their major sales channels. Each account rep had created their reports. Some of the more IT savvy had created very powerful pivot charts on top of DB views whereas others were copy-pasting data from PDF reports. Generalizing the best version and sharing it across the board accelerated the other 7 accounts without really creating a lot of development overhead.

6.     Be picky with your ETL

Typically, the single biggest expense (in terms of work hours) for a typical BI initiative is ETL – the process of extracting data from a diverse set of sources, cleaning it up and then loading it into a structure more suited for reporting. Most ETL work is “below the surface” and it’s hard to justify significant effort in the work stream without raising alarm bells.  We would question ETL investments very carefully against the value they are likely to deliver.

At a high level, the ETL process usually consists of several key challenges:

–          Connecting to a diversity of data sources

–          Scrubbing the data clean of errors and inconsistencies

–          Formatting it to a normalized enterprise view (in the data warehouse)

A plethora of “best-practices” exists for each of these areas which ensures that the design is most efficient in terms of s performance, scalability, extensibility etc. These documents are usually written by technical authors who rarely look at the cost-benefit analysis of each of their design decisions. It is critical for the project teams to view things from that lense otherwise teams will spend weeks importing and cleaning artifacts that are rarely used.

One recommendation we provide that usually stirs a heated debate with the “best practice” police is to build views on top of OLTP data instead of moving data physically. This is particularly useful with low-volume dimensional data.  A similar recommendation is to consider using natural keys instead of surrogate keys wherever possible. This saves the effort of having to merge in and version data every time it changes however comes at the price of a lack of standardization across the entire warehouse.  We are by no means suggesting that this should be done in all cases, merely that wherever possible, a shorter, cheaper, and less performant approach be considered as a design option.

This is easier said than done of course and will require the teams to re-think many aspects of the project. Priorities will come into question and a new perspective will needs to be developed that puts a dollar amount (both short and long term) against design decisions. Solutions exist for all these but what we are highlighting is the need to question your vendor’s “best practices” against what is “best for you”.

7.     Save on Licensing – don’t aim for the best and greatest.

BI product offerings have been increasing over time new vendors and new paradigms getting introduced in the past few years. There is a wide range of options available now and prices are being driven down by competition. Selecting the products for your technology platform can be daunting. Popular choices for SME’s fall in three major categories:

Cloud options – There are a lot of factors to consider but the key one, in my opinion, is the CapEx/OpEx debate. If you are in a cash-strapped industry, the cloud will work out nicely for you.

Open source – Things have improved there significantly however choose wisely here, the Total cost of ownership remains a challenge in the available BI toolset. There is great work on many areas like big-data where open-source wins hands down but my opinion is those are rarely relevant to the SME’s problem.

Paid options – Don’t choose the specialized products. They might be the best at what they do but that’s usually all they do. SME’s will usually use 20% of a product’s toolset, 80% of the time. Rather than looking for the best of breed, look for the product that gives you the 20% you need at the lowest possible cost. Nearly all tools are extensible and what it doesn’t provide can usually be achieved through customization.

Specialized tools require rare (expensive) consultants and they need effort in trying to make the different specialized pieces to work together. For an SME this is usually not worth the effort.

8.     PAD your schedules… no really!

If your business has been running without a formal BI infrastructure for a while, the odds are it can survive without it a little longer. When planning projects, plan with a little pad. This is contrary to typical project management paradigms which will tell you to finish as quickly as possible to keep overall costs low. In the SME BI implementation we are talking about, this doesn’t always make sense because:

1. Things aren’t always completely planned out. Most companies I have worked with aren’t sure of what they want till they start looking at it. Business users will think of new requirements and change existing requirements almost as soon as they start seeing the system.
2. You will always run into project dependencies that you have no control over. This ranges from things that need approval in a monthly executive meeting to servers that aren’t available when you need them to be. With each of these delays you are burning expensive consulting time.
3. Cheaper resources (especially those based offshore) have a limited time overlap with onsite resources. So issues that occur after 10am for example will rarely get caught till the next day.

Both of these issues mean that most things will be a stop-and-go exercise with re-prioritizations happening all the time. The way to avoid this is by having more than one work stream going at all times. If one is stuck for any reason, switch your resources to the other. Keep enough pad in your customer’s expectations to allow for this shuffle.

10. Pick a good partner

This seems obvious. If you are an SME looking to implement a BI project, you’re not claiming to be an expert in the area. Having someone on board who knows the way seems obvious – picking the right one, maybe not so obvious. In my opinion, in a low budget situation, there are a few things to look out for (other than the usual):

–          Look for someone with experience working in your industry. Warehouse modeling is perhaps one of the most critical parts of the BI exercise and probably one of the smallest (in terms of time spent in the overall SDLC). A good understanding of your industry’s typical data model will limit the requirement analysis to “differences from the industry” rather than exploration from the ground up.

–          Look for product expertise. Toolsets change every couple of years. New trends show up with all the vendors that change how things work. Look for someone who understands the importance of making technology choices not just from a technology fascination perspective but from an ROI perspective. The uber-product expert usually has very little interest in whether CapEx oriented solution is better for the company versus an OpEx based one – the primary argument in cloud based services. Make sure the company you engage has both kinds of people and shares the goal of delivering bottom line ROI.

–          Blend your teams across shores. This topic has fallen off of favor in the recent past but can reduce your costs between one-half and one-third. Most of the big 5 consulting firms have great resources that cost them no more than 15-20$/hr internally. By the time they get priced to the customer it is more like 80$/hr. Going to these vendors directly can get you five-star delivery without the five-star price tag. Offshore markets have evolved significantly with small specialized units showing up on the maps that focus on very narrow areas with the BI landscape. Finding them is definitely a worthwhile exercise.

Conclusion

Our top 10 are based on 10 years of implementation BI for small companies with big goals. Big goals that BI can help achieve. BI projects have long had the reputation of “much bigger than expected” – we feel solutions exist today to change that. Our top 10 is an attempt to synthesize the key things to watch out for. We’d love to hear back if you find these useful.

About the Author

Muhammad Omer is a partner at Allied Consultants (www.alliedc.com) and works on building Business Intelligence and Systems integration solutions for small and medium business across the globe. He has over 11 years of experience working in the North American and Middle-eastern markets working for medium and large scale enterprises. He may be contacted at [email protected] for any comments or feedback on the topic.