Full Security Integration with the BI Platform

Data Services 4.0 now leverages the BI Platform for security and administration. This puts all of the security for Data Services in one place, instead of it being fragmented across the various repositories. Now, Data Services repositories are managed through the Central Management Console where the administrator can set rights on individual repositories just like he would with any other object in the BusinessObjects world.

This change brings some welcomed improvements. For example, users now log in to the Data Services Designer or Management Console with their BusinessObjects user ID. You no longer need to enter database credentials when you log in to Designer. Instead, once you log into BusinessObjects, you are presented with a simple list of Data Services repositories to choose from.

Built in Text processing transform

I first saw the BusinessObjects Text Analysis offering in a live demo years back. There was a ton of potential, but it was yet another tool separate from the rest of the suite and never got much momentum. Now that technology is included in Data Services 4.0. Using Text Processing, Data Services can read unstructured text such as e-mails and extract the important information from it, including names, places, dates, and their relationships. I am excited to see how this will be used by our clients.

New Datastore support for HANA

A lot of hype has surrounded SAP’s new High-Performance Analytic Appliance, or HANA software. This technology will allow for in-memory analysis of data and provides real-time data loads from any data source that Data Services supports. Many of the BI client tools in BusinessObjects 4.0 support HANA, including Xcelsius Dashboard Design and Explorer. I’m looking forward to the chance to work with this new product and find out what it can really do.

64-bit Server Architecture

Just like BusinessObjects 4.0, Data Services 4.0 has native 64-bit support.  Both the server components and client tools are now 64-bit.   The 32-bit server components are no longer offered, but there is still a 32-bit Designer tool available.

Get More Information

Data Services 4.0 is currently in ramp-up with SAP.  If you are interested in what Data Services 4.0 has to offer, read more about it from SAP’s What’s New guide.  Data Services 4.0 is scheduled for general release in June 2011.

No related posts.

I recently published the post 5 Reasons Why You Should Use an SAP BusinessObjects Universe to highlight some of the core benefits you’ll see when implementing the semantic layer for you organization.  The post was derived from speaking with organizations new to the Business Objects world about the benefits of leveraging the semantic layer. Throughout those talks, I also get to hear a lot of misconceptions about SAP BusinessObjects Universes.  Sometimes they are born out of a simple lack of understanding of the SAP BusinessObjects architecture, while other times they are propagated by competing business intelligence vendors.  (Shocking, I know.)

In any case, I’m here to set the record straight.  While the universe semantic layer isn’t perfect, it is pretty good at what it does and keeps getting better.  Here are some common misconceptions that I’ve heard over the years.  Did I miss any?  Let me know in the comments…

Universes affect query performance by adding unnecessary overhead.

False. In its simplest form, a universe is merely a semantic layer that maps your database objects to standard business terminology to provide an intuitive way to work with information.  When queries are refreshed, the business terminology is turned back into SQL and executed on the appropriate database directly.  If your query has three items in it, so will the resultset coming back from the database.  There aren’t any middleware components or services that add overhead to this process.

The Universe is simply a cube of my data, formatted for reporting.

False. (See above.)  The data returned by your universe continues to reside wherever it does today.  The universe semantic layer simply defines how those tables are joined, how the objects are defined, etc…

I need to have a data warehouse or data mart to be able to use a Universe.

False. While query performance will generally be better if running against a schema designed for reporting (i.e. data mart/data warehouse), as a technology Universes are equally at home pointing directly to data contained in CRM, ERP and other transactional systems.  We often use this method when implementing operational reporting, where our client needs real-time access to their raw transaction information.

I can’t use a Universe because I want to aggregate by different dimensions depending upon the report.

False. When developing a universe, you simply define how objects should aggregate – sum, avg, count, etc… -  not specific dimensions by which to aggregate.  By properly using SQL Aggregation and Measure Projection in the definition of your universe objects, the result set will dynamically aggregate at whatever level is appropriate given the dimension objects you selected.  If you select ‘Year’ and ‘Total Sales’, the query will return one row for each year showing total sales.  Add ‘Product Line’ to the query, and now you’ll have one row per year per product line, etc…  All of this done automagically by the universe.

Our application uses a proprietary database, so it won’t work with a Universe.

False (sort of). In general, as long as the vendor provides an ODBC driver for the database you should be able to access it’s information through the universe using a Generic ODBC connection.  However (and it’s a big however) the functions that you can use in the universe are limited to those that are supported by the ODBC driver.  If you’d like to roll up all the sales by year and the driver doesn’t support a Date function to do that, you’re out of luck.  (Obviously you could still do these conversions at the report level, but that’s a different topic as we’re talking specifically about universe functionality here.)

At any rate, this lack of support has more to do with the specific ODBC driver than the universe, as you would run into the same issue no matter how you tried to access that data.  In these cases, we typically evaluate the reporting requirements alongside the capabilities of the ODBC driver itself before making the decision to move forward with the generic driver, or work towards getting the data into a more standard database platform.

Question:  What misconceptions have you had/heard about SAP BusinessObjects Universes?

Related posts:

1. 5 Reasons Why You Should Use SAP BusinessObjects Universes
2. What is a Business Objects Universe?
3. Exploring 4.0: Information Design Tool

In meeting with organizations who are new to SAP BusinessObjects or who are legacy Crystal Reports users, we are often asked the question “Why should I use a Business Objects Universe?”.  To organizations who have worked with universes for a period of time, the advantages and value proposition are clear.  But for organizations who are new to universes, it can often be a challenge to quantify the value of the semantic layer.

In this post I hope to outline a few of the reasons why we recommend implementing universes, even if you’re not using an SAP BusinessObjects tool that specifically requires them.

Integrated security

One of the most common ways that security is implemented into an SAP BusinessObjects Universe is to dynamically restrict data being returned by a query.  In this example, only sales belonging to the user that ran the report or viewed the dashboard would be shown.  You can also use this technique to segregate data by customer, regional manager, product division, warehouse, etc… to give everyone a personalized view of their information.  But that’s just the beginning.

You can also use the built-in universe security to restrict access to particular objects (such as salary or SSN), control the types of queries a user can run, change the database connection depending on the user, and even dynamically map to other tables.  Leveraging the universe model gives you complete security over your environment, and the best part is you only have to apply the security once and it takes affect whether the user is refreshing a canned report, viewing a dashboard or creating their own ad hoc report.

Single version of the truth

We’ve all been there.  You’re at a meeting discussing sales numbers and three people give three different answers on what last month’s sales were.  The Sales Manager defines it as bookings made.  The Warehouse Team counts only orders that were physically shipped out the door.  And Finance, well they pull a bunch of different information into their spreadsheet and come up with a third version.  They may all be correct, but they’re also all wrong.

Building a universe layer “forces” you to come together and define the terminology that you use, perhaps in the case above creating separate “sales” and “shipments” measures to satisfy everyone’s reporting requirements.  When reporting through the universe, everyone in the organization will benefit by having access to the same information, defined in the same way, using the same calculations.

Build once and reuse

Some of the clients we meet with, particularly those that have been using Crystal Reports for some time, are resistant to using universes because “it would be easier to just hardcode the SQL” or “our data structure is too complex”.  Sure, it may be easier the first time to hardcode a SQL query into a report, but what about the second, fifth or tenth time you have to do that?  And as for the data being too complex, with few exceptions if you can define it in SQL you can build it into a universe.  This ability to define the universe semantic layer one time and leverage it across the entire SAP BusinessObjects toolset is huge.

Maintenance/change management

Perhaps one of the most overlooked benefits of utilizing the universe layer is around maintenance and change management.  With the universe model all of your logic is in one place, making maintenance straightforward.  Plus, if the data model for any of your upstream systems changes, you can in most cases modify the universe to reflect those changes without having to touch any of your downstream reports, dashboards or analytics.  Just imagine if you had 2,000 reports written against hardcoded SQL sitting on top of a data model that has now changed radically in a new release.  I’ve seen it happen, and it wasn’t pretty.

Intuitive interface for developers and end-users

This is what it’s all about: providing end-users and developers alike with a drag and drop interface where they could build reports using business terminology rather than SQL code.  Which is more intuitive to an end-user: SUM(DB_AdvWorksDW.dbo.FactResellerSales.SalesAmount) or ‘Sum of Reseller Sales’?  Now imagine this simple example using CASE statements, datatype conversions, etc… and you’ll understand the true power of the SAP BusinessObjects universe.

In using the universe layer, report authors and developers can reference information in common business terminology, and be shielded by all the technical complexity and transformations that are happening behind the scenes.

Want to learn more?

There are many benefits to using the Universe model, these are just a few of the most popular.  Want to learn more?  Call us at (877) 844-7476 or email info@alteksolutions.com to see how you can leverage the power of the universe for your organization.

Question:  Does your organization utilize SAP BusinessObjects universes?  If so what advantages do you see, and did you find any downsides?  If you don’t use universes, why not?

Related posts:

1. Top 5 Misconceptions About Business Objects Universes
2. Maintain Peak Performance With a Comprehensive SAP BusinessObjects Health Check
3. Five Reasons Why Ad Hoc Reporting Won’t Work in Some Organizations

Following SAP’s big acquisition of BusinessObjects, more and more companies running SAP are looking to BusinessObjects as an answer for their reporting needs. The BusinessObjects tools WebIntelligence and Xcelsius are well suited for this role. However, getting SAP data into those tools is not yet as easy as SAP would like.

Tools like WebIntelligence and Xcelsius are primarily driven by the BusinessObjects universe. Unfortunately for SAP customers, SAP has not yet produced a universe connector to allow you to tap directly into your SAP R/3 data. Instead, SAP customers need to leverage an ETL tool to extract the SAP data into a relational database. For a variety of reasons, BusinessObjects Data Services is the tool of choice for this job. In this post I will explain how Data Services talks to SAP to extract data.  

Data Services connects to SAP using the same information you would use to connect through the SAP GUI tool.  In Data Services, you create a datastore with all the required SAP server information.  Once set up, you can import SAP tables into Data Services just as you would with any other data source.  Instead of using a standard Data Flow as you would with a relational database, with SAP data you must use the R/3 Data Flow.

The standard method for manually extracting SAP data is to write custom ABAP code which generates a flat data file. The flat file can then be consumed by other processes.  With the R/3 Data Flow, Data Services uses this same method, although it automates the entire process.  A basic SAP R/3 Data Flow in Data Services will generate ABAP on-the-fly and upload it to the SAP server via a shared directory.  The shared directory can be a network share or FTP that both the Data Services and SAP servers can access.  Data Services then asks SAP to execute this ABAP code.  The resulting flat file is generated in the same shared folder, which Data Services then processes.

Inside an R/3 Data Flow -- Joining 4 Sales fact tables from SAP R/3

From here, any normal Data Services transformations can occur on the data — the extraction from SAP is complete. The SAP data can be joined with data from other sources, or you can use Data Services’ built in Key Generation and Table Comparison transforms to build a true datamart with slowly changing dimensions and delta-loaded fact tables. The BusinessObjects universe can be laid on top of this database, allowing you to use all the other tools in the BusinessObjects suite — reporting with Web Intelligence, analysis with Explorer, or dashboarding with Xcelsius.

SAP also offers several “Rapid Marts”, which combine Data Services jobs with a prebuilt universe, providing a plug-and-play solution. There are Rapid Marts for Sales, Inventory, Cost Centers, Projects, and more.  Whether you choose to use one of these prebuilt solutions or design your own, it is good to understand the challenges involved with connecting BusinessObjects to SAP.

For More Information

• SAP Community Network Wiki on BusinessObjects and SAP
• SAP BusinessObjects Rapid Marts
• Altek Solutions Data Management Services

No related posts.

Back in March I posted “Five Reasons Why Ad Hoc Reporting Won’t Work In Some Organizations“.  In that post, I listed some of the major concerns that I hear from potential clients when the subject of ad hoc reporting is broached.  As promised, I will be addressing each of those concerns and showing how they can be overcome.  We’ve already addressed “Our people aren’t smart enough to create their own reports.“  Now let’s turn our attention to data, and look at “We need to massage the data first.”

Why Reinvent the Wheel Each Month?

A few years ago we worked with a client who was brand new to the world of Business Intelligence.  They had a very small IT staff, and as such their IT Director was the one responsible for reporting information back to the Executive Team.

I asked the IT Director to take me through his reporting process so I could understand the workflow a bit better.  He started by opening a few Excel spreadsheets, showed me how he does a copy and paste from their various mainframe systems into Excel, and then how he massages the data.  All told, this process took him about 14 hours each week — just to get this information to the rest of the Executive Team.

Absent any real data integration platform, this process seemed to be a good alternative for where they were as a company at the time.  But as he dashed through Excel renaming dimensions here, altering data there…it was clear that manipulation aspect was consuming most of his time.  I asked him why they didn’t fix the data at the source system, and he commented that it was just easier to fix the data here each month rather than making changes to the source system, retraining the people entering the data, etc…  Now that might be alright for a one-time analysis, but why spend countless hours each month changing the same data in the same way?

Touch and Cleanse the Data Once

If this story sounds familiar, it should.  Virtually every client we meet has some degree of data quality issues.  Without a plan and routine maintenance, companies may find themselves having to deal with bad data on a continual basis, which can derail the success of any business initiative.  (I won’t even begin to go into the compliance issues that can arise if data is constantly being manipulated in Excel.)

Depending upon the operational system, you may want to implement transactional data quality and controls, where the data is validated as it’s being entered into the system.  For others, it may make more sense to batch scrub the data as part of your nightly processing.  Or, more likely, you’ll implement a hybrid of both approaches.

Whatever approach is taken, the key is to implement a process that allows you to touch and cleanse that data once.  Then it can be confidently made available to all your downstream systems without having to involve Excel in the process.  By ensuring that your organization has a solid data quality foundation, you can be confident that all your efforts are based on accurate and complete data.

Question:  What data quality challenges do you have in your organization?  What have you done to address them, and what was the outcome?

Related posts:

1. Why Ad Hoc Reporting Won’t Work (part 5): If I Give Them Ad Hoc Access to Data, They’ll Keep Asking For More
2. Why Ad Hoc Reporting Won’t Work (part 3): I don’t want people poking around in data they shouldn’t have access to
3. Why Ad Hoc Reporting Won’t Work (part 4): I Pay People in IT to Create Reports

The evolution of Enterprise Information Management (EIM) has brought us to several strategies. Now, there is consolidation, federation, or a combination of the two. Business Objects offers solutions in the form of two products: Data Integrator and Data Federator.

Data Integrator is the classic ETL (Extract, Transfer, Load) tool that has a graphical user interface to consolidate your data. Data Federator is an EII (Enterprise Integration Information) tool that creates a virtual database to “federate” multiple data sources. Deciding on a potential strategy (or strategies) will be based on how and when the data will be accessed as well as development and monetary resources

Data Integrator (Consolidate)

Data Integrator excels at creating data marts/warehouses that include historical data, i.e. if the data needs to be archived from the ERP system. Usually, the data is processed on a nightly basis and brought over into a defined data model that can be reported against using a universe or other SQL based tool. Because this structure has been defined during design, performance should be addressed with the use of indexes, summary tables, etc.

Functional Overview of Business Objects Data Integrator

While time and money is needed to invest in developing and deploying a data warehouse, it is well worth the investment. Transforming and loading data into a warehouse or mart via a tool like Data Integrator will provide high performance and trusted, accurate data for reporting and dashboards.

Pros

• Allows for historical storage and archiving
• Data moved ahead of time (batch)
• Moves predetermined set of data
• Designer for systematic development
• Distributed batch processing and parallelism
• Aggregates disparate data from multiple source systems
• Handles large volumes of data
• Manages long and complex calculations
• Excellent query performance

Cons

• Long loading times could be encountered
• More expensive to develop (time and money)
• No ad-hoc data capabilities
• Real time data access can be more costly to attain

Data Federator (Federate)

Data Federator excels at a quicker implementation (thereby more cost effective), real time data access, and on-demand data access. Data Federator creates a virtual database using several different data sources, such as Oracle, SQL Server, MySQL, and text files. With this virtual database, there isn’t a need to worry about additional storage for the data, and the performance is handled by the Data Federator Query Server engine.

Functional Overview of Business Objects Data Federator

If time and money are an issue, as they often are, Data Federator can also provide a faster turnaround time for your ROI. With incremental deployment, multiple data sources can be tied together into a virtual database that can be used for both reporting and dashboards. Using the transforms available through the Designer tool and the Query Server, high performance and trusted data can be achieved as well.

Pros

• Access current data stored in sources
• Data moved on-demand (at query time)
• Only requested data returned
• Real Time data access
• Quicker development time – supports incremental development
• Optimized query engine – bind, hash, merge joins for efficient processing
• Reduces data storage and data transfer

Cons

• Still does query against original sources
  • Unlike Data Integrator that uses a specific BI database(s) or data warehouse.
• Only contains as much data as source system
  • If data is archived off source, data is no longer available in Federator.
• Query performance is good, but not as quick as a data warehouse

A Hybrid Approach – The Best of Both Worlds

While it seems that Data Federator could be used to replace ETL, in reality, it works best when used in conjunction with Data Integrator. Data Federator can be used as a prototype or first phase of data warehouse and solve connectivity issues, but it cannot make queries run faster on OLTP databases. Data Integrator with Data Federator can provide full data integration.

Using Business Objects Data Federator to add real-time or detailed data to an existing data warehouse.

For example, if the warehouse is used to store the historical data, Data Federator can virtually merge the warehouse and the ERP system to provide both historical and real time data (as seen in the picture below). This model includes the best of both worlds: storage, transformation, and performance through Data Integrator and real-time access and performance via Data Federator.

Using Data Integrator and Data Federator together ensures that your Business Intelligence users have trustworthy and timely information for your organization to make business decisions. Both of these tools are now available for your organization to determine the best strategy to deliver this information.

Related posts:

1. Why Ad Hoc Reporting Won’t Work (part 2): We Need to Massage the Data First