Thursday, January 24, 2013

Starting OBIEE 11g Services

OBIEE 11g comes with fancy enterprise manager, and weblogic console, however if you have to restart the server for any reason, the services may not start back up on their own (unlike 10g). The checklist below will be helpful if 11g fails to start

1. Start the node manager:

/MiddlewareHome/wlserver_10.3/server/bin/nohup ./startNodeManager.sh &

2. Set WLS (weblogic server) environment:

cd /MiddlewareHome/user_projects/domains/bifoundation_domain/bin

Setting the environment:

./setOBIDomainEnv.sh
./setDomainEnv.sh

3. Start the WLS:

nohup ./startWebLogic.sh -Dweblogic.managment.username=weblogic -Dweblogic.management.password=webdev123 &

Weblogic logs are located at:

/MiddlewareHome/user_projects/domains/bifoundation_domain/servers/AdminServer/logs
The server log file
/MiddlewareHome/user_projects/domains/bifoundation_domain/servers/AdminServer/logs/AdminServer.log

change auto boot file:
/MiddlewareHome/user_projects/domains/bifoundation_domain/servers/AdminServer/security/boot.properties (this is the file as in nohup.out log)
/MiddlewareHome/user_projects/domains/bifoundation_domain/servers/bi_server1/security/boot.properties

4. Login to WLS console:

http://localhost:7001/em

and navigate to :

Farm>WebLogic Domains>bi_cluster.

Click control > Start

4b. Use following command to start BI application from command line :
cd /MiddlewareHome/instances/instance1/bin

./opmnctl status

./opmnctl startall

error log
/MiddlewareHome/instances/instance1/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1/console~coreapplication_obips1~1.log

This can be also started from EM

By now, services should be up and running. If not, check log files for error messages and you will need to do troubleshooting.


LOG FILES:

-The server log file. All server side log events will be written to this file:
/MiddlewareHome/user_projects/domains/bifoundation_domain/servers/AdminServer/logs/AdminServer.log

-Presentation server log:
get /MiddlewareHome/instances/instance1/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1/sawlog0.log


CONFIG FILES:
-Instance config:
/MiddlewareHome/instances/instance1/config/OracleBIPresentationServicesComponent/coreapplication_obips1

-NQSConfig.ini file:
/MiddlewareHome/instances/instance1/config/OracleBIServerComponent/coreapplication_obis1/

Saturday, January 19, 2013

Top 10 Costly Mistakes for Business Intelligence / OBIEE Implementation Projects

Top 10 Costly Mistakes for Business Intelligence / OBIEE Implementation Projects

Okay, you have done your research, you have meticulously compared market offerings, and lined up a team of resources for implementation for Oracle BI 11g product. Senior management is supporting you, and they are expecting a great implementation of the product from your team. With a company new to the tool, how to make sure of a successful BI implementation? Having over a decade of experience in the industry, I assure you, success doesn't come easy. Selecting a technology stack is less that 10% of the battle fought to achieve your overall BI goals and expectations. Below are listed some of the most common and highly avoidable mistakes by various organizations from tiny to gigantic in sizes:

1. Not setting the expectations right with users and senior management - Having millions of dollars of funding approved from management does not mean that ton of reports will be available for analysis. It is best to release small batch of reports first, and then build upon it based on the users feedback. IT department is often under pressure to arrive being a hero group, unfortunately, there is no easy button in BI.

2. Going for BI Apps and assuming all out of the box reports will be ready and used by business - The Out-of-The box BI apps are extensive, and look good. But I am yet to find a company, that has used them without major customizations. Most Out of the box report will require some level of customization in order to be able to be used. BI apps make sense to purchase to get ETL code, which will help in building warehouse tables and having a baseline repository to begin with. Even that would require some customizations, matching the same done to source systems. Verdict - consider BI Apps to get ETL code and a baseline warehouse built. But plan to perform customizations.

3. Not spending enough time on designing repository to enable intuitive ad-hoc reporting - One of the key selling features of product like OBIEE 11g is the ad-hoc reporting capability, which allows end users to make simple changes to reports themselves, thus reducing their dependence on IT, and cutting down development cycle. For this to happen, the data model should support it and repository should be in line with ad-hoc reporting. Without proper foresight, this may not happen, and is a costly mistake to be avoided from the beginning. Some companies plan for this, however during the development, with several changes coming back and forth, and multiple fix and break cycles, its easy to loose track. Change in BI projects will be there, it cannot be eliminated, it has to be managed keeping ad-hoc in mind. One way to reduce change cycles is to involve users from beginning, and this brings us to our next costly mistake as below.

4. Lack of user involvement during development project - User know what they want, in their heads, unfortunately, they cant communicate in BI language. They will communicate in the language of MS Excel or whatever current reporting tool they have. Waiting for six months or closed to delivery date, to show first cut to the business would be a big mistake, as they are new to BI. Their light bulbs will go on only after looking at the reports and dashboard. Best would be to do a proof of concept (POC), or to involve them from the beginning, by having working sessions, or letting them see work in progress in development. I kid you not, this a common mistake.

5. Not testing subject areas for Ad-hoc reporting - Developers and rest of the team are under pressure often to deliver the laid out requirements, which usually is a list of reports. Seldom, the subject area itself is part of that list. This is a big mistake, and letting SA go to prod without testing will add to user frustration during ad-hoc reporting (this is supposed to be a key value offering of BI tool set).

6. Not having BI standards for requirement gathering, development, testing and migrations - Including inconsistent naming conventions across subject areas. Too much focus is instead spent on technology stack setup, users requirement at hand, and deliverable on which success will be evaluated. Although standards evolve over time, its best to set a baseline from the beginning. Your users will be happier and less frustrated 5 years down the road. It is a good idea to have a resource who is knowledgeable and has access to BI best practices for development and implementation.

7. Hiring a great data modeller experienced with relational source systems, and letting him lead dimensional model design - will you visit a brain surgeon for a root canal treatment? No I suppose, then why would you go to a relational data modeler for dimensional modeling project? Yet, it is a common mistake made. It can lead to wrong designs, which brings to our next mistake.

8. Using relational models instead of dimensional model - Technically - possible, practically - useless. Warehouse models should be dimensional - star and snowflake - period. Reasons? Relational models are good with write and read systems, while dimensional models are best for read performance. Relational models offer poor performance, are difficult to maintain, complex to troubleshoot in case of issue, very hard to enhance per customer request, and most of all, it does not support ad-hoc reporting.
9. Not having application level backups or other faster ways for disaster recovery. Most companies have teams that will backup all servers and have a plan for disaster recovery. However, you would be surprised, how much time a read disaster will take for recovery. If you loose one file to a bad sector in hard disk, will business accept waiting 48 hours (or more), before server is back up? Application level back ups would be a minimum, if fail over servers or high availability environment is not setup. It will allow at least the application admins to do basic restore of services in event of a disaster.

10. Having developers or power users write access to non ad-hoc reports in Test, or Production environment -
This is a trap, to make changes in any environment other than Dev (ad-hoc reporting is a valid exception although, where reports are built directly in production by end users). Unless you want to be wondering down the road about which environment has the latest working code, no one should be making changes to test or prod servers. Not even administrators, leave alone power users. I'm not kidding, it is a common mistake. Delivery deadline, and customer push, makes toughest of the security policemen bend over to this rule. Set the expectations right, with proper reasoning, and avoid future headache.


I feel compelled to point out one more item, which may or may not be a mistake depending on who you are talking to, so let me continue:

11. Having multiple installations of software withing the same organization - There are some merits of this, like HR in most cases would want a silo installation and have a silo development team due to sensitive nature of data they own. They have a separate hardware and software stack. But finance and sales having silos doesn't make sense. This is particularly true with large shops, who don't care about chump change it takes to build a silo, and fall for flexibility, independence and speed of development with this model. For most of us, it is a mistake to avoid.

Hope you found this article useful.

Friday, January 18, 2013

OBIEE 11g Upgrade considerations

OBIEE 11g is a major leap in technology when compared to its prior version 10g. To go for newer version is a big decision, and required correct timing and planning. Following are important considerations when planning this move.

OBIEE 11g upgrade required revising hardware, annual license costs as weblogic is additional component used in 11g technology stack, and maintenance costs.

1. New installation Vs Upgrade - For companies going for Oracle BI for the first time, 11g is the only choice. Oracle is going to discontinue its dupport for 10g by Mid of 2014, so that leaves 10g implementation out of considerations.
2. Volume of published reports in current version- If the current set of reports are large, this will impact the upgrade timeframe for 11g. Managers need to plan sufficient time and budget for post upgrade bug fixes to published reports in 10g, redesign for reports that cannot be simply fixed due to change in application logic (this should be minimum), testing by development and users community.
3. Upgrade to 11g will be impacted by volume of customizations in 10g platform, like use of Java componenet for reporting.
4. Technical capabilities of administration team - 11g works on weblogic, which is pretty complex if things go wrong and need to be fixed. Although under normal circumstances, only working knowlegde of weblogic should suffice.
5. Security model - 11g has completely new security model. Catalog level and RPD level security is supported for backward compatibility only, and its recommended to migrate to weblogic based application roles based security. Depending of number of groups and users, this might be a challenge to complete
6. Retesting of entire upgrated catalog is a must - to ensure none of the existing functionality is lot.
7. Requirements for new features - if there is an upcoming project that will need integration with essbase, or has ragged hierarchies, its much better idea to upgrade before starting that project to take advantage of new features.
8. Development team and end user training - 11g interface is quite different than 10g, so some level of training is required for sure.
9. Cloud based 11g implementation - instead of installing and maintaing inhouse, implementing on a internet based cloud server might be a good option for those who dont want to maintain a in-house administration team. Although it will not be very flexible when it comes to migrations, and the model will not be cost affective when size of implementation increases.

I hope the above helps in making a choice, to read about all installation steps, please my blog post on  page below:

http://obiee911.blogspot.com/2013/01/obiee-11g-installation.html

OBIEE 11g Installation

You might be thinking, how come I blogged for OBIEE uninstallation, before OBIEE installation. Well, I had to do atleast one uninstall before finally we were able to have a successfull install, and so I followed the same order while blogging too. Here are the steps we followed:

Environment:
Server OS: Linux red hat -
Client OS: Win XP Proff
Database (for BI repository): Oracle 10g
BI Warehouse database may or may not be the same, it does not matter.

Requirements:
- Linux terminal (putty) and FTP access
- root or sudo root access in Linux
- sysdba access to database (just DBA access will not work)
- All install / patch files downloaded and copied to Linux server
- RCU (repository creation utility) must be downloaded and saved to Client machine.
- Either firewall software must be disabled on Linux box, or all ports required by BI server must be opened before installation



Steps for Installation:

1. Run the RCU on windows client machine (or linux if you have 32 bit server). This installs two schemas on database which store BI server metadata. OBIEE 11g merged tables for BI publisher, usage tracking and iBots all into this schemas, along with several new tables to support BI functionality. This step will fail if you dont have a database ID with sysdba level of access.
  • Run rcu.bat file from command prompt
  • Select ‘Create Repository’, click Next
  • Enter the database connection details where schemas will be installed
  • Ignore the warning about the AL32UTF8 character set (not sure who should not ignore)
  • Select the components to be installed and specify a prefix, example 'DEV'. We installed two schemas - MDS and BIPLATFORM
  • Click Next all the way till its completed
2. On Linux server, create a group 'orainstall', a user 'obi'  and a folder as 'MiddlewareHome'. Following commands will comein handy:
  • groupadd oinstall
  • /usr/sbin/user add obi -g oinstall
  • mkdir -p /MiddlewareHome
  • chown -R obi:oinstall /MiddlewareHome
  • chmod -R 775 /MiddlewareHome
3. On your client machine, download and install a X-windows server like X Cygwin or Xming. Google this for free installable software. This will be useful to run installation from your desktop. If you have direct access to Linux box, and dont need to use putty, skip this step and go directly to step 8 below.

4.  Login to Linux using terminal and execute command
              xhost + <>
This step will let you view GUI on your windows machine from installer running on linux server (compared to text based terminal / putty window). I am not aware if Linux has something similar to remote desktop connection as in windows. So this should work till we have something like RDC.

5. Open Putty on your windows machine and specify the host name for linux server. Next, click on
         Connections > SSH > X11
    and check 'Enable X11 forwarding'. Also make sure 'MIT-Magic-Cookie-1' is selected in radio button below.

6. Login to linux server using Putty and execute following command
        export DISPLAY= <>
    Replace IP address with that of your windows client machine.

7. Make sure that your windows client is not accepting the connections. To do that, execute command like "xterm" or "firefox" in Putty, and it should open a window for application on your local client. That way, then you start the installation for 11g, the GUI window will open on your windows machine.

8. In putty window, change directory to the folder containing the installer (bishiphome/Disk1)

9. Execute command “./runInstaller”

10. Select the ‘create local inventory option’, when asked to create the oracle inventory.

11. Choose “Enterprise Install”

12. Enter password for weblogic user. Follow minimum password standard, alphanumeric, atleast 8 characters long. Then click Nexk.

13. Browse or enter Oracle Middleware Home Location (/MiddlewareHone). This should fill out rest of the options for this screen. Hit Next.

14. Select the components you want to install. Not many companies have Real-Time Decisions, so uncheck it if you dont have the license, to save some space. Hit Next

15. On this screen enter database details for BI Platform schema that was created in step 1 above followed by MDS schema in next step.

16. Choose Auto Port Configuration on the next screen

17. Specify email ID on the next screen

18. Hit "Install" to start the installation process.

Thats it. Unless you have space limitations, you should be good with installation. Follow my blog for more articles if you liked this post. Give me a google+ for your appreciation. Thank you.





  

OBIEE 11g Certification Exam

To achieve OBIEE 11g certified profession status, looks like there are two levels of OBIEE 11g certifications available:

First: Exam ID 1Z0-591 - Oracle Business Intelligence Foundation Suite 11g Essentials
  • 120 mins
  • 75 Que
  • 63% passing score (48 questions)
  • The exam questions are based on version OBIEE 11.1.1.5


   
Docs are available: http://docs.oracle.com/cd/E23943_01/bi.htm

The doc link is not going to help much, unless someone wants to read 2.5 GB of documentations. Following topics might help to prepare for exam (ofcourse experience should make up for lack for complete reading):

Exam Topics

Oracle BI and EPM System Overview                                                                                                         
·         Describe how OBIEE addresses business intelligence challenges
·         Describe the Oracle Business Intelligence Enterprise Edition (OBIEE) products
OBI EE Architecture
·         Describe major components of the OBIEE architecture
·         Describe the function of each component
Oracle BI Installation
·         Describe the Installation Architecture for OBI EE
·         Explain system requirements, supported platforms and installation pre-requisites
·         Install OBIEE
OBI EE User Interface Overview
·         Describe the components of OBI EE User Interface
·         Perform Oracle Analysis to construct, modify, and save BI requests
·         Build Dashboards
·         Create BI Delivers
Dimensional Modeling
·         Describe the differences between Transactional and Analytical Systems
·         Describe Data Warehousing
·         Build Dimensions, Facts and Hierarchies
·         Explain Dimensional Modeling
Oracle BI Repository Overview
·         Describe the three layers of the OBI Repository
·         Use the Administration Tool to view and modify the repository
Build Basic Repository
·         Define a System Data Source Name (DSN) for a data source
·         Build the Physical layer of the Repository
·         Build the Logical layer of the Repository
·         Build the Presentation layer of the repository
Repository Testing and Deployment
·         Describe techniques for testing a repository
·         Validate and test a repository
Complex Repository Modeling
·         Describe Derived Metrics and its use
·         Configure Derived Metrics based on existing logical columns
·         Configure Derived Metrics based on physical columns
Advanced Repository Modeling
·         Create Data Partitioning
·         Create Segmentation of Data
·         Create Time Series Data
Building a Repository with Multiple Sources
·         Add multiple sources to an existing logical table source for a dimension in the business model
·         Add a second table source to a dimension in the business model
Hierarchies and Level Based Measures
·         Create a dimensional hierarchy
·         Use level-based measures
·         Create a rank measure
·         Describe Aggregate tables and the purpose they serve
·         Create Aggregate tables in the OBI repository
Using Variables
·         Explain session variables
·         Describe repository variables
·         Describe initialization blocks
·         Implement a dynamic repository variable
Implementation Methodology and Best Practices
·         Describe the implementation phases
Oracle Business Intelligence Enterprise Edition Security
·         Describe authentication and authorization
·         Describe the different levels at which security is enforced
·         Configure security using Security Manager
·         Configure Query Governing controls
Cache Management & Performance Tuning
·         Explain the purpose of cache in OBI EE
·         Configure caching and manage cache files
·         Develop caching strategies
·         Describe the request and data flow in OBI EE
MS Office Integration
·         Use MS Office Plug-In for OBI EE
·         Describe the required components to enable office integration
·         Install and configure MS office server and client for OBI EE
Spatial Integration
·         Explain Spatial Intelligence and Integration
·         Add Map based visualization to OBI EE
Write Back within OBI EE
·         Describe Write Back capabilities
·         Configure Write Back with OBI EE
Essbase Integration
·         Implement Essbase as a Data Source
·         Configure BI Server Metadata for Essbase
·         Explain Essbase Integration

Next Level exam is:
 
Oracle Business Intelligence Foundation Suite 11g Certified Implementation Specialist

 http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=458&get_params=p_track_id:OBIFS11gOPN

The certification covers skills such as: installing OBIEE, building the BI Server metadata repository, building BI dashboards, constructing ad hoc queries, defining security settings and configuring and managing cache files. The exam targets the intermediate-level implementation team member. Up-to-date training and field experience are recommended.