Tibco Loglogic® Unity User`s Guide

Transcript

TIBCO LogLogic® Unity User's Guide Software Release 1.0.0 November 2014 Document Update: January 2015 Two-Second Advantage® 2 Important Information SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THE EMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANY OTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE. USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS AND CONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTED SOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THE CLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOAD OR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE) OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USER LICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THE SOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, AND YOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BE BOUND BY THE SAME. This document contains confidential information that is subject to U.S. and international copyright laws and treaties. No part of this document may be reproduced in any form without the written authorization of TIBCO Software Inc. TIBCO, LogLogic, and Two-Second Advantage are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/or other countries. Enterprise Java Beans (EJB), Java Platform Enterprise Edition (Java EE), Java 2 Platform Enterprise Edition (J2EE), and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle Corporation in the U.S. and other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only. THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOT ALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASED AT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWARE VERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM. THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCO SOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME. THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY OR INDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES. Copyright © 2014-2015 TIBCO Software Inc. ALL RIGHTS RESERVED. TIBCO Software Inc. Confidential Information TIBCO LogLogic® Unity User's Guide 3 Contents TIBCO Documentation and Support Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Signing into the Web UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Signing out of the Web UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Editing your Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Search Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Using Content Assist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Using the Search field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Using the Time field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Search Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 About Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Raw Data Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Table Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Correlation Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Search Syntax Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Event Query Language Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Common Search Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 USE Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 FILTER Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Predefined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Time Range Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 COLUMNS Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 GROUP BY Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 SORT BY Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 LIMIT Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Search Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Event Correlation Language Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Rule Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Identifier Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Event Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Aggregation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Having Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Correlation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 TIBCO LogLogic® Unity User's Guide 4 Correlation Blok (ECL) Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 About Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 About Filter Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 About Correlation Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Viewing All Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Adding a Blok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Modifying Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Deleting Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 About Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Viewing All Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Adding a Time Blok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Modifying Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Manage Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Manage Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Viewing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Adding a Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Configuring SMTP Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Editing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Synchronizing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Enabling/Disabling Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Deleting Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Monitor Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Viewing Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Acknowledging Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Viewing Alert Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Viewing Event Group Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Manage Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Viewing Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Adding a Source Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Enabling/Disabling Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Editing Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Deleting Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Supported Log Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Filter Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 TIBCO LogLogic® Unity User's Guide 5 TIBCO Documentation and Support Services All TIBCO documentation is available in the TIBCO Documentation Library, which can be found here: https://docs.tibco.com Product-Specific Documentation Documentation for TIBCO products is not bundled with the software. Instead, it is available on the TIBCO Documentation site. To directly access documentation for this product, double-click the following file: TIBCO_HOME/release_notes/TIB_logu_version_docinfo.html The following documents for this product can be found in the TIBCO Documentation site: ● TIBCO LogLogic® Unity Installation and Configuration Guide ● TIBCO LogLogic® Unity User's Guide ● TIBCO LogLogic® Unity Developer's Guide ● TIBCO LogLogic® Unity Tutorials How to Contact TIBCO Support For comments or problems with this manual or the software it addresses, contact TIBCO Support as follows: ● For an overview of TIBCO Support, and information about getting started with TIBCO Support, visit this site: http://www.tibco.com/services/support ● If you already have a valid maintenance or support contract, visit this site: https://support.tibco.com Entry to this site requires a user name and password. If you do not have a user name, you can request one. How to Join TIBCOmmunity TIBCOmmunity is an online destination for TIBCO customers, partners, and resident experts. It is a place to share and access the collective experience of the TIBCO community. TIBCOmmunity offers forums, blogs, and access to a variety of resources. To register, go to: http://www.tibcommunity.com TIBCO LogLogic® Unity User's Guide 6 Overview LogLogic® Unity is a sleek, modern, and scalable platform enabling technical teams to resolve open issues which require advanced troubleshooting techniques, complex root cause analysis, or deep forensics. LogLogic Unity is a Log processing Search & Alerting tool that takes data from any source and structures that data. This allows for intuitive, fast, and complete interaction with data, resulting in faster turn around from open to close in issue resolution. Its powerful Web User Interface (UI) enables fast and flexible searching, correlation, and alerting. This provides operational insights into infrastructure and application performance and security events. Key Features ● Modular search queries – Use all or part of saved search filters to build new search queries using new building Blok technology. ● Multiple search queries – Run multiple searches at the same time. ● Working data sets – Work with multiple search results without losing what you are working on. Walk away and come back without losing your search results. ● Data lookup – Enrich your experience with lookup tables enhancing search and alerting capabilities. ● Data at rest correlation – Perform advanced correlation against historical data to identify trends. ● Data in motion correlation – Maintain advanced correlation in memory to identify key patterns for alerting. ● Comprehensive APIs – Leverage core functionality using intuitive APIs built on Representational State Transfer (REST). ● Scalable clustering technology – Scale horizontally as needed to maintain performance and storage. LogLogic Unity architectural view is shown in the following illustration: TIBCO LogLogic® Unity User's Guide 7 Signing into the Web UI The Web UI enables fast and flexible searching, correlation, and alerting. Prerequisites You must start the LogLogic Unity system before logging into the Web UI. For details, refer to the TIBCO LogLogic® Unity Installation and Configuration Guide. Procedure 1. Open a browser and navigate to the URL http://localhost:9680, where localhost is the default hostname and 9680 is the default port number. 2. In the Sign in window, enter your credentials. The default user name is admin and the default password is admin. On successful authentication for the first time user, the product walk-through screens are displayed. Click Next to continue. Click Try it out to open the Search tab and run a sample query using the sample source configuration. The Search tab is opened showing the sample query in the Search field. Click the sample query. Click to view results of to start a new search, see Search Basics for details. TIBCO LogLogic® Unity User's Guide 8 Signing out of the Web UI Procedure 1. Click located on the upper right corner on the main header. 2. Click the Sign out link. When you are successfully signed out of the system, the Sign in window appears again. TIBCO LogLogic® Unity User's Guide 9 Editing your Profile You can update your own profile at any time. Procedure 1. Click located on the upper right corner on the main header and select the Edit profile link. 2. Update the information in the corresponding fields. You can update the email address, personal information, phone number, and password. Only an Admin (a user with Administrator privileges) can update the user ID once it is created. 3. If you update the password in the Old password field, enter the same password in the New password field. 4. Click Save to save the updated information. After updating the information, the Sign in window appears. You must sign in using the updated credentials. TIBCO LogLogic® Unity User's Guide 10 Search Basics From the Search tab, you can easily interact with your data. It allows you to run simple and complex searches, save search elements and time ranges in the form of Bloks, retrieve results to analyze failures or other anomalies. The basic search retrieves all events that match the search term. Advanced searches are retrieved by a "pipeline" concept, where expressions are separated by pipes ("|"). LogLogic Unity search query language is intuitive and efficient, allowing you to search large data and view results in seconds. The search query mainly supports three types of languages: Structured Query Language (SQL) dialect, Event Query Language (EQL) and the Event Correlation Language (ECL). The Search Syntax Reference helps you understand how to form a search query. By default, results are returned in the ascending order. The Search and Time fields can be combined (AND-ed) or used alone as described below: ● If you define time period in the Search or Time field, the results are retrieved for the specified time period. ● If you define time in the Search field and Time field, the results are retrieved for the intersection of the time periods. ● You must specify time in either in the Search or Time field. All dates and times are defined in the local time zone of the machine where the system is installed and it is not based on the browser time zone. For complex queries, you can create different types of Bloks that can be reused in future searches. For detailed information about how to build and use Bloks, see About Bloks. For sample search examples, see Search Examples. Click to add multiple search tabs. You can run multiple searches using different search elements on the same data to analyze any anomalies. After you run a query, the header and search fields are collapsed to maximize the results view and the search query is displayed on the top. Click on the top right corner to expand header and search fields. located Using Content Assist The Content Assist feature shows typeahead or contextual matches and completions for each keyword as you type into the Search field. These contextual matches are retrieved from your data. You can get assistance for language syntax, column names, source configuration names, recent search history, and Blok names. To enable the Content Assist feature, click located on the upper right corner on the main header and select the Enable Content Assist link. A checkmark indicates that the content assist feature is enabled. As you start typing in the Search field, the Content assist panel displays: ● Suggestions help you build your search query by suggesting the next matching term. ● Matching terms identify the matching word as you continue to type. ● Sources allow you to define a source configuration to be used in your query. ● History displays all recent search entries that you can choose from to run a query. TIBCO LogLogic® Unity User's Guide 11 Click on the term to select and add it in the Search field. Once you finished adding all terms, select the fragment in the Search field, the Save fragment as Blok button gets enabled. Click Save fragment as Blok to save the statement as a Blok for later use. The Add new Blok window appears. For instructions on how to add a Blok, see Adding a Blok. Using the Search field You can enter any valid combinations of syntax languages (SQL or EQL or ECL) with source, filter, or regular expressions. You can use single or multiple terms. Enter USE to start an EQL statement and SELECT to start an SQL statement. You can search based on Bloks. For details on how to add a new Blok or use the existing Blok, see About Bloks. As you start typing, the Content Assist feature shows contextual matches and completions for each keyword into the Search field. If the search expression syntax is valid, a green checkmark next to the syntax. Click appears to view results. For example, enter the use system | sys_eventTime in -1d:NOW in the Search field to retrieve events from the system source configuration profile within a certain time range. Using the Time field You can enter absolute and relative time ranges. You can search based on Bloks. From the Search tab, enter the time period in the Time field and click new time Blok or use the existing Blok, see About Time Bloks. . For details on how to add a All dates and times are defined in the local time zone of the machine where the system is installed and it is not based on the browser time zone. For example, enter -5h to retrieve all events that occur in the last 5 hours. TIBCO LogLogic® Unity User's Guide 12 Search Results After running a search query, you can view search results in the Result tab. You can visualize results using Charts or Data panel. After running a query, if you retrieve lots of results, you can group the results without having to issue a new query, and then drill-down into the information. This allows you to see both aggregated counts as well as create visualization elements to better isolate trends and issues. You can include multiple filters to narrow your results. Create a filter in the context of a message and view results based on a specific filter. After running the search query, a progress bar appears above the Result tab showing the number of processed events. Based on your data, it may take a few minutes to retrieve results into all three panels. By default, a maximum of 10,000 results will be displayed in the Result tab. To increase the limit, use the LIMIT clause in your query. See LIMIT Statement for details. Add multiple result tabs to view the same data in different forms. Click to add multiple result tabs. When results are grouped together, a new Result tab displays showing the grouped results for the selected value. The Result tab is divided into three panels: ● Charts - displays results graphically using a line chart in the top panel. ● Columns - provides all available columns and their associated values based on each search query in the left bottom panel. ● Data - displays data in different formats in the right bottom panel: raw format and normalized tabular format. TIBCO LogLogic® Unity User's Guide 13 Charts A chart is a visual representation of your data. By using elements such as lines (in a line chart), a chart displays a series of numeric data in a graphical format. You can add multiple result tabs to view the same data in different formats. A chart displays the total event count for a specified time period. You can use different options to view chart details, zoom in and out of the chart, show/hide chart panel. From the Charts panel, you can do the following: ● Show/hide Charts panel Click the icon located at the upper right corner of the Charts panel to show the charts panel. Click to hide the Charts panel. ● Zoom in/out of Charts You can zoom in or zoom out of a particular area of chart using the time-range picker. Grab the handles on the X-axis time-range picker, it turns into a slider. Drag the slider across the Xaxis to define the time range that you want to zoom in. A chart is updated for the selected time. The following Line chart displays the zoomed in data for a specified time range and the Data panel shows the filtered results for the corresponding time range. You can expand and collapse the time range by dragging the borders of the selected time range to the desired location. Once you define the time range, position the mouse inside the selected time range and drag the slider to define the new time range. Similarly, you can define a specific time by clicking on the chart. The time range can be adjusted at any time. As you adjust the time range on the chart, the Columns and the Data panels are adjusted automatically for the selected time range. TIBCO LogLogic® Unity User's Guide 14 ● View Chart details Hover your mouse over a certain area of the chart to view the details. ● Filter results based on the time range You can fine-tune your search results based on the time range. Click the event count (the line that represents the number of messages) on the chart or define the time range by zooming in on the chart to view results in the Data panel. A new filter is added for the defined time and the filtered results are displayed on the Data panel. Columns Based on your search query, all available columns are displayed in the Columns panel. You can group together your results based on any column and the value associated with that column. Similarly, filtering helps you fine-tune your search results when analyzing big data. By default, three system columns are displayed to show results in the Data panel. For a list of system columns, see About Columns. TIBCO LogLogic® Unity User's Guide 15 From the Columns panel, you can do the following: ● Show/hide Columns panel Click panel. ● located on the right corner to hide the Columns panel. Click to show the Columns Find columns You can quickly find the desired column by typing the column name in the Find field. As you start typing a column name in the Find field, all possible columns that start with the letters that are typed get displayed in the pane. The Columns panel is refreshed based on the selection. ● Show or hide columns from the Data panel Select the check box to show the column in the Data panel. Clear the check box to hide the column from the Data panel. Click Select all to select all columns. Click Deselect all to hide all columns. The located on the left side of the column name defines that the column is displayed in the Data panel. The Data panel gets updated immediately based on your selection. ● Expand column list Click located at the bottom of the column list to expand the column list. It allows you to view all columns retrieved from the data. TIBCO LogLogic® Unity User's Guide 16 ● View column value details Click the column value and then select Show values to view the details of the selected value. The window displays a maximum 100 distinct values for the selected column. The Percent column is calculated using the maximum 100 distinct values. When the distinct values for a column exceeds 100, the Percent column is not displayed. If you filter on a particular column-value, then the percent value on the top will show the percentage of occurrence of this particular column-value in the entire result set. The following illustration displays values for the column sys_bodySize. TIBCO LogLogic® Unity User's Guide 17 ● Group by values Click the column value and then select Group by to view grouped results. A new Result tab appears showing the results that are grouped by the column. The following illustration displays results grouped by the column sys_bodySize. You can group by different time ranges. Click the timestamp value, and select the Group Dates by option. From the list, select the option to group your results by different time periods. A new Result tab appears showing the results that are grouped by different time units. TIBCO LogLogic® Unity User's Guide 18 You can aggregate columns that have Integers and Long values. Click the column value and select Add aggregation. Define how to group values in the aggregation column. The options are: SUM, MIN, MAX, AVG. A new column is added in the Data panel. The following illustration displays a new aggregation column (SUM) gets added in the Data panel. About Columns The system (event metadata) columns are indexed so searching is faster on the system columns. All system columns are displayed with the prefix sys_ and all columns from built-in parsers are displayed with the prefix ll_ in the column list. The following list describes all system columns in the LogLogic Unity event. Name Type Description sys_eventTime Timestamp The UTC time of the event in Epoch milliseconds. sys_body String The text of the message. sys_bodySize Integer The size in number of bytes of the body. sys_sourceType Integer TIBCO LogLogic® Log Management Intelligence (LMI) device type ID. TIBCO LogLogic® Unity User's Guide 19 Name Type Description sys_collectIP InetAddress The IP from where the event originated. This must support both IPv4 and IPv6. sys_collectTime Timestamp The UTC time of the event when it was ingested into the LogLogic Unity event storage. sys_filename String The file name for event collected from a file. sys_fileLineNumber Integer The line number in file. sys_tenant String The customer identifier. sys_domain String The customer sub-identifier. sys_partition Long The identifier of the portion of the data on the data node. sys_offset Long The location in the LogLogic Unity event store. sys_eventKey String A unique key that refers to an event in the LogLogic Unity store. sys_lmiEventKey String A unique key that refers to an event in the LogLogic LMI event store. sys_lmiApplianceId String An identifier for the LMI appliance. sys_lmiDomain Integer, String The LMI Domain is a component of the LMI device (source) identifier. sys_sourceDns String The DNS name for the event_source_ip. Data Based on your search query, the retrieved data is displayed in the normalized tabular format. Each event is summarized per row. You can view data in the following three formats: ● Raw Format ● Table Format ● Correlation Format TIBCO LogLogic® Unity User's Guide 20 From the Data panel, you can do the following: ● View event count The total number of retrieved events is displayed on the top right side. ● Filter your results You can create a filter using the column value and message body text to fine-tune your search results. Click ● to show or hide filters from the Data panel. Add a new source configuration You can add a new source configuration that can be activated to analyze results in the Data panel. Click located on the top right side of the Data panel to add a new source configuration. For instructions on how to add source configuration, see Adding a Source Configuration. ● Download your results You can share your search results with others. Click located on the top right side corner of the Data panel to download search results in the CSV format. Raw Data Format Based on your search query, the results are displayed in Raw data format. Each message is summarized per row. The same result set can be viewed in the Table format. Using the Raw data format, you can do the following: The column value options are displayed in the following illustration. ● Show/hide columns from the Raw data Click the Columns on / off link to show selected columns below the message or to hide columns to view messages in the raw format. ● Wrap long messages Click the Wordwrap text on / off link to indicate if long messages should break at normal word break points or to display long messages. TIBCO LogLogic® Unity User's Guide 21 ● Filter data Click the column value and select Include this Filter to filter the data based on the value. If you select Exclude this Filter, the results will exclude the specified value. The Data panel displays results immediately based on the defined filters. You can add multiple filters to fine-tune your search results. The following illustration displays the Raw data showing filtered results for the sys_bodySize: 611 value. Click to show or hide filters from the Data panel. Click the column value and select Include this filter on Result tab to filter the data based on the value in a new Result tab. If you select Exclude this filter from Result tab, a new Result tab will display results excluding the specified value. The filters that are used in the current Result tab are not carried over in the new Result tab. You can filter based on the message body. Drag the mouse to select the message body and select Include this filter to filter your results based on the message body filter. If you select Exclude this filter the results will exclude the specified message body. ● Sort columns You can sort on any column, including group-by count(*) column, group-by aggregation-columns, and other columns. Click the column value and then select Sort Ascending to sort columns in the ascending order. Click the column value and then select Sort Descending to sort columns in the descending order. TIBCO LogLogic® Unity User's Guide 22 ● Group by values Click the column value and select Group by to view grouped results. A new Result tab appears showing grouped results for the selected value as shown below. You can group by different time ranges. Click the timestamp value, then select Group Dates by option, and then select the option to group your results by different time periods. The Raw data view is refreshed showing the results that are grouped by defined time period. ● Hide columns from the Raw data Click the column value and then select Hide to hide the selected column from the Raw data format. Table Format Based on your search query, the results are displayed in normalized Table format. Each message is summarized per row. The same result set can be viewed in the Raw data format. Using the Table format, you can do the following: TIBCO LogLogic® Unity User's Guide 23 ● View Messages in the Table format Click the Messages on / off link to show or hide the message body. Alternatively, hover over the message number link to display the message body. TIBCO LogLogic® Unity User's Guide 24 ● Filter data Click the column value and then select Include this Filter to filter the data based on the value. If you select Exclude this Filter the results will exclude the specified value. The Table view displays results based on the defined filters immediately. You can add multiple filters to fine-tune your search results. The following illustration displays the Table showing filtered results for the sys_sourceType: 65536 value. Click to show or hide filters from the Table panel. Click the column value and then select Include this filter on Result tab to filter the data based on the value in a new Result tab. If you select Exclude this filter from Result tab, a new Result tab will display results excluding the specified value. The filters that are used in the current Result tab are not carried over in the new Result tab. You can filter based on the message body. To do this, make sure that the Messages on link is selected. Drag the mouse to select the message body and select Include this filter to filter your results based on the message body filter. If you select Exclude this filter the results will exclude the specified message body. The following illustration shows results based on the message body filter Logon. TIBCO LogLogic® Unity User's Guide 25 ● Sort columns You can sort on any column, including group-by count(*) column, group-by aggregation-columns, and other columns. Click the column header and then select Sort Ascending to sort columns in the ascending order. Click the column value and then select Sort Descending to sort columns in the descending order. TIBCO LogLogic® Unity User's Guide 26 ● Group by values Click the column header and then select Group By to view grouped results. A new Result tab appears showing the grouped results for the selected value as shown below. You can group by different time range options using the Group Dates by option. Click the time value, then select Group Dates by option, and then select the period to group your results by different time periods. The Table panel is refreshed showing the results that are grouped by defined time period. ● Hide columns from the Table Click the column header and then select Hide to hide the selected column from the Table panel. Correlation Format The correlation search results are displayed every time the rule's conditions are met. A correlation Blok is created from a simple correlation rule. For detailed information on how to define correlation rules, refer to Event Correlation Language Reference. For detailed information about correlation Bloks, refer to About Correlation Bloks. 1. Type the correlation rule in the Search field. Alternatively, click located next to the Search field and select Choose Blok and then select the correlation Blok from the list. TIBCO LogLogic® Unity User's Guide 27 2. Enter the time period in the Time field and click . The correlation results display all events that contributed to the triggering of the correlation rule. Based on the correlation rule, the columns (correlation events and event groups) are extracted in a table format. Each row helps you analyze the associated values of the columns and event groups. The following illustration displays the defined correlation rule in the Search field and retrieved events in the Charts, Columns, and Data panels. 3. Click the event count link to view the event details in a new Search tab. As shown in the above illustration, click 19 (the event count link), the new search tab opens with the auto-generated EQL query in the Search field for the events associated with that event count. The Charts, Columns, and Data panels display the results associated for that event count as shown below. TIBCO LogLogic® Unity User's Guide 28 Search Syntax Reference LogLogic Unity search query language is intuitive and efficient, allowing you to search large data and view results in seconds. The search query mainly supports three types of languages: Event Query Language (EQL), Structured Query Language (SQL) dialect, and Event Correlation Language (ECL). Both EQL and SQL are equally capable for searching, but syntaxes are different in some cases. For example, simply providing a string in EQL is understood as a full text search, but it will give a syntax error in SQL. So the translation is not always literal. EQL is easy to use, however, SQL is more familiar and scripting is easy using existing SQL tools. Using EQL, you can define filters, regular expressions, sources, time ranges. ECL is useful to find patterns in a given set of data and used for correlation purposes. Event Query Language Reference The search query supports two types of query languages: Event Query Language (EQL) and LogLogic Unity Structured Query Language (SQL) dialect. The EQL query is composed of different parts separated by pipe ( | ) character. The pipe delimiter is used to separate the expression and each subsequent expression. Each pipe-delimited expression further processes search results from the preceding expression. For more structured queries, subset of SQL is supported that is mainly focused on SELECT statement. Both languages can be used interchangeably, all that is available in EQL can be achieved via SQL and vice versa except the following two differences: ● EQL supports the full text search statement, but SQL does not support this statement. For details, see Filter Statement. ● Multiple EQL filter expression statements, separated by a pipe, get automatically combined using the AND operator into a single filter expression. SQL does not support this feature. The EQL and SQL language rules are based on a Backus-Naur Form (BNF)-like notation as shown below: ::= ; where, ● non-terminal symbols in syntax rules have angle brackets (< >). For example, in the rule ::= "+" ; the is a non-terminal symbol and the rule specifies that as an expression is the addition of any number of integers. ● terminal symbols are shown in double quotes (" "). For example, the "+" in the previous example. ● as an additional shortcut notation to BNF, optional symbols (that can occur zero or one times) are followed by a question mark (?). For example, in the rule ::= (ASC| DESC)?; a column name used for sort is a column name optionally followed by the keywords ASC or DESC. ● optional symbols that can occur zero or any number of times are followed by an asterisk (*). For example, in the rule ::= ("," )*; an itemList can contain one or more comma separated items. ● multiple symbols are grouped together using parenthesis ( ) when some common operation is applied, for example, the selection of one member of the group, or to indicate that the entire group can be repeated zero or more times. An example is shown in the previous bullet item. ● words that are all capitalized represent keywords (special terminal symbols). For example, the keywords ASC and DESC in the column name for sort described in the previous example. All parts of the query are optional but the overall the syntax is as shown below: ::= ("|" )* ; TIBCO LogLogic® Unity User's Guide 29 ::= | | | | | ; String literals and identifiers (including keyspace, column family names, and source configuration names) are case-sensitive but all EQL keywords are case-insensitive. For example, 'USE Windows' and 'use Windows' are treated in the same way. String literal can be quoted with single (') or double (") quotes. The quotes (single or double) inside the string literal has to be prefixed with backslash ( \ ) character. The \ character change to be prefixed with another backslash ( \\ ). For example, "Mike's car" or 'Mike\'s car'. A special syntax for time range can be used. For details, see Time Range Expressions. In this syntax reference topic, EQL keywords in UPPERCASE are used as a convention for easy readability. Examples Expression Definition sys_sourceType = 65536 and sys_eventTime Events from source type '65536' in last 5 days, display result as a table with columns sys_eventTime, sys_collectIP, ll_eventStatus, and ll_type. in -5d | columns sys_eventTime, sys_collectIP, ll_eventStatus, ll_type use Microsoft_Windows | severity = 'Critical' and user = "Fred" and sys_eventTime in -1M | group by vm Using the source configuration Microsoft Windows, display results of all critical events for a given user per virtual machine in last month. columns count(*) Common Search Commands LogLogic Unity uses the following search commands. Command Definition USE Defines the event sources which includes the parsing configuration. For details, see USE Statement. COLUMNS Defines which columns should appear in search results. For details, see COLUMNS Statement. GROUP BY Groups search results based on specified columns. For details, see GROUP BY Statement. SORT BY Sorts search results based on the expression. For details, see SORT BY Statement. LIMIT Limits the size of search results to be displayed. For details, see LIMIT Statement. For detailed information about filters, see Filter Statement. TIBCO LogLogic® Unity User's Guide 30 USE Statement A source is the name of the log source from which a particular event originates. The source defines which logs from which log sources to parse, how to parse them, and what column to extract in order to execute this query. The USE defines the event log sources which includes the parsing configuration. This is an optional parameter but it is recommended to improve performance by reducing the set of event sources and set of parsers used. ::= "USE" ( "," )* ; The USE statement consists of the USE keyword followed by one or many source configuration names separated by commas. An is a letter followed by any sequence of letters, digits, or an underscore (_). If you do not specify any source configuration in the Search field, the results will be retrieved in this order: first all enabled LogLogic built-in source configurations, next all enabled log sources that are non-LogLogic specific but have source filters defined, and lastly the system log source. The user defined source configurations without the source filter will not be included in the search query. For a detailed list of built-in source configurations, see Supported Log Sources. For more information about source configurations, see Manage Source Configurations. Examples Source Configuration Expression Definition use Windows The result will display all events from Windows sources. use Windows, Cisco The result will display all events from Windows and Cisco log sources. FILTER Statement A filter is an expression that specifies the conditions that events must satisfy to be returned by this query. The filter criteria can be in form of free text search of the entire body or value of a particular preparsed or parsed column. The system (event metadata) columns are indexed so searching is faster on the system columns. The list of available columns is determined by list of event sources. In case the list of event sources are not available, the system will do the best to extract those columns using heuristics algorithms. For queries, the filter should contain a time condition, otherwise the default is used. A filter statement is any expression that evaluates to a result of type boolean. Any event that does not satisfy this condition will be eliminated from the results. An event satisfies the condition if it returns true when the actual event values are substituted for any variable references. The following table explains the types of filter statements that can be used. For the complete syntax, shown as a BNF grammar, see Filter Syntax. TIBCO LogLogic® Unity User's Guide 31 Statement / Operator Description AND Narrows your search results by only returning those events where each one of the AND conditions evaluates to true. For example, use AND to return results containing all specified keywords. When AND is used, the results contain all specified keywords and do not contain entries with just one of the specified keywords. OR Expands your search results by returning events where either of the OR conditions evaluates to true. For example, use OR to return results containing any and all specified keywords. OR is ideal when you have common synonyms for a keyword. To narrow results as much as possible, combine OR statements with AND statements. Full text search Full text search on the body of each event can be performed by simply providing the phrase that needs to be enclosed in double quotes. For example, use system | "authentication failed" will retrieve all events that contain the above phrase. The EQL full text search (specifically on sys_body) is exactly the same as the SQL LIKE statement on the sys_body (so "use system | 'Bob'" is exactly the same as "select * from system where sys_body LIKE 'Bob'"). Equals (=), Not equals (<> ), (!=), Lower than (<), Lower or equal (<=), Greater than (>), Greater or equal (>=) Plus (+), Minus (-), Multiply (asterisk (*)), Divide (forward slash (/), A comparison condition compares two expressions using the operator specified in the comparison, which may be one of seven possible comparison operators with well known meanings. The comparison condition evaluates to true only if the comparison condition is satisfied. This may be used to narrow search results. For example, "col1 > col2/100". The arithmetic (+,-,*,/) and string concatenation (||) operators can be used to create parts of other conditions. For example, "column1 + column2 < 5" or "col3 * 4 - 1000 > col5" String concatenation (||) TIBCO LogLogic® Unity User's Guide 32 Statement / Operator Description Function Any of a set of predefined functions. For details, see Predefined Functions. They can be only used in Filter expressions or as part of Source Configuration expressions. The parameters of the functions can be expressions themselves and will be evaluated before the function is called. For example, "ToInt(col1 + col2)" will add the contents of the columns of the event named col1 and col2, and pass the result to the ToInt function and the result of the function will be used. BETWEEN Narrows your search results by only selecting those events where the left hand side expression evaluates to a value that is between the two right hand side target expressions. Supports Timestamps, Long, and Integers. For time range syntax details, see Time Range Expressions. IN Narrows your search results. Checks if value matches any one of the values in a set. For example, "eventID IN ('id1', 'id2', 'id3')". Supports all data types. For time range syntax details, see Time Range Expressions. IS NULL, IS NOT NULL Narrows your search results by accepting or rejecting the event based on whether the evaluated expression is null or not null. An expression most frequently becomes null if a column named in the expression has no value for the current event. Supports all data types. TIBCO LogLogic® Unity User's Guide 33 Statement / Operator Description LIKE, NOT LIKE Expands your search results. Returns true if it matches the supplied pattern. The sys_body column is special, because the supplied pattern is used to do a full text search on the event body. For all other columns, the following rules are used to interpret the supplied string. ● The character percent (%) is the wildcard character (matches zero or more characters). ● The character underscore (_) means that it matches exactly one character. ● The backslash character (\) is used to escape itself and the above two characters if a literal search for any is desired. The LIKE statement for columns starting with sys_ uses a full text search, not SQL LIKE syntax. Since string literals in EQL/SQL require backslashes (\) to be escaped, note that additionally escaping for the LIKE statement doubles the escaping requirement. The simple rule to follow is to construct the match string using the above rules, then simply double up each backslash. The following examples show the actual syntax ( not the escaping needed for Java): ● col1 LIKE "a_b" - produces a match for "acb", "adb" and so on ● col1 LIKE "a\\_b" - produces a match for "a_b" but not "acb". Note the double backslashes. ● col1 LIKE "a\\\\_b" - produces a match for "a\cb" and "a\db" ● col1 LIKE "a%b" - produces a match for "ab", "acb", "accb" and so on ● col1 LIKE "a\\%b" - produces a match for "a %b" but not "acb" TIBCO LogLogic® Unity User's Guide 34 Statement / Operator Description REGEXP, NOT REGEXP Narrows your search results. Returns true if it matches the supplied pattern. The pattern syntax uses POSIX syntax. Since string literals in EQL/SQL require backslashes (\) to be escaped, note that all the backslashes inside a regular expression pattern must be doubled up, similar to the LIKE statement. Examples: ● col1 REGEXP "[a-z]b" - produces a match for "ab", "cb" but not "Ab" or "_b" ● col1 REGEXP "\\w*" - produces a match for a word, for example "this" or "that", but not "this and that" Examples Filter Expression Definition "Authentication" and sys_eventTime in The result will display all events that contain Authentication from last 1 year. -1y use sample | ll_sourceUser = 'SiteSvrAdmin' | sys_eventTime in '2014-02-02' The result will display all events that contain column 'll_sourceUser' and value is 'SiteSvrAdmin' on the 2nd of February 2014. Predefined Functions The functions that are available in the EQL are listed below. The smart list functions are usually used in filter expressions and source configuration. The conversion functions are typically used when adding a new source configuration, when you need to define new columns, where the expressions for new columns can use conversion functions to convert between data types and combine them using various operators. For instructions on how to add a new source configuration, see Adding a Source Configuration. Function Name Arguments Returns (String 1, String 2) The value associated with String2 in the smart list named String1. Smart List functions lookup Example: lookup("list1", "key1") or $list1("key1") TIBCO LogLogic® Unity User's Guide 35 Function Name Arguments Returns length (expression) Returns the length of the string value of the evaluated expression. If the expression is not a string, for example, an integer, it will convert it to a string first. Example: length("abc") is 3, length(3145) is 4 (after converting the integer 3145 to the string "3145") Conversion functions ToTimestamp (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) The expression, which should evaluate to a string, is interpreted as a time according to the supplied formatString. If the conversion fails, null is returned, unless a default string is provided, which is interpreted as a time and returned. Example: ToTimestamp( logFileStringTi mestampField, "dd, MM, yyyy HH:mm:ss", "America/ Los_Angeles", "01, 01, 1970 00:00:00") If timezone is omitted or is empty, the system default timezone is used. ToIP (expression) or (expression, defaultValue) Convert the expression to an IP address (Java InetAddress). If the conversion fails, null is returned, unless a default string is provided, which is interpreted as an IP address and returned. Example: ToIP(ipAddressField, "www.oracle.com") TIBCO LogLogic® Unity User's Guide 36 Function Name Arguments Returns ToTimestampString (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) Same as ToTimestamp, except it gets converted to string to get a printable timestamp. Example: ToTimestampString(timestamp , "dd, MM, yyyy HH:mm:ss", "America/Los_Angeles", "01, 01, 1970 00:00:00") If timezone is omitted or is empty, the system default timezone is used. ToInt (expression) or (expression, defaultValue) The obvious conversion to integer with default value taken if not convertible. Example: ToInt("1348") or ToInt(numberField, 0) ToLong (expression) or (expression, defaultValue) The obvious conversion to Long with default value taken if not convertible. Example: ToLong("1348") or ToLong(numberField, 0) ToString (expression) or (expression, defaultValue) The obvious conversion to String with default value taken if not convertible. Example: ToString(124.5) or ToString(numberField, "null") ToFloat (expression) or (expression, defaultValue) The obvious conversion to Float with default value taken if not convertible. Example: ToFloat("1348.2") or ToLong(numberField, 0.0) ToBool (expression) or (expression, defaultValue) The obvious conversion to Boolean with default value taken if not convertible. Example: ToBool("FALSE") or ToBool(col1, FALSE) Time Range Expressions The time range for IN operator understands both relative time as well as absolute time. Absolute time is the same as in BETWEEN operator. Relative time is defined as: , for example: -5d means 5 days ago. TIBCO LogLogic® Unity User's Guide 37 All dates and times are defined in the local time zone of the machine where the system is installed and it is not based on the browser time zone. The following time units are available: ● s - second ● m - minute ● h - hour ● d - day ● w - week ● M - month ● q - quarter (3 months) ● y - year The supported timestamp formats are: ● Any day of the week; for example, MON, TUE, WED, THU, FRI, SAT, SUN ● NOW specifies up to the current time ● Today specifies as the end of the day (23:59:59) ● yyyy-MM-dd HH:mm:ss ● MM/dd/yyyy HH:mm:ss ● BETWEEN and IN supports also dates (yyyy-MM-dd or MM/dd/yyyy), the interpretation depends if it is used as beginning or end of time period. When used in beginning it is equivalent to yyyyMM-dd 00:00:00, when used at the end - yyyy-MM-dd 23:59:59. Examples Time Range Expression Definition -5d Last 5 days including today. -1M Last month. "2014-10-20" From 2014-10-20 00:00:00 and 2014-10-20 23:59:59. "2014-10-20":"2014-10-25" From 2014-10-20 00:00:00 until 2014-10-25 23:59:59. "2014-10-20 14:00:00":"2014-10-25 From 2014-10-20 14:00:00 until 2014-10-25 20:00:10. 20:00:10" "2014-10-20 14:00:00":NOW From 2014-10-20 14:00:00 until now (the time the query was issued). MON:NOW From beginning of last Monday till the current time. TIBCO LogLogic® Unity User's Guide 38 COLUMNS Statement COLUMNS is used to define which columns should appear in the results and how they should be computed. ::= "COLUMNS" | ; ::= ( "," )* ; ::= ( "," )* ; A COLUMNS statement can be the name of a column or it can be an aggregate function formed from column values. The following data types of columns are supported: ● String ● Integer ● Long ● Double ● Boolean ● Timestamp ● IP address If all columns use aggregation functions, the result will contain only one row with results of the aggregation. For grouping details, see GROUP BY Statement. Examples Columns Expression Definition columns sys_eventTime, ll_collectIP, The result will be a table with 3 columns: sys_eventTime, ll_collectIP, sys_body. The columns could be one of the pre-parsed columns like sys_eventTime, sys_body etc. or columns from configured parsers. See USE Statement. sys_body columns count(ll_sourceUser) The result will have one column with one row with count of all events that has ll_sourceUser column with no empty value. GROUP BY Statement Grouping can be used to group values by given column or columns. Grouping requires list of grouping columns and the list of aggregation columns. A group by expression can be a column name, and optional list of aggregation functions after COLUMNS keyword. ::= "GROUP BY" ( "," )* )? (COLUMNS ("," )* )?; The following aggregation functions are supported: ● COUNT(*): Count all the rows ● COUNT(columnName): Count all the rows in which the value of the column is not null ● COUNT(DISTINCT columnName): Count all distinct values from the column TIBCO LogLogic® Unity User's Guide 39 ● SUM(column): Sum all values from the column. Supports numerical types (Integer, Long, Double) ● AVG(column): Provide average value for the column. Supports numerical types (Integer, Long, Double) ● MIN(column): Smallest value of the column. Supports all data types that can be ordered (Integer, Long, Double, Timestamp, String) ● MAX(column): Largest value of the column. Supports all data types that can be ordered (Integer, Long, Double, Timestamp, String) ● DURATION(timestamp): Returns the difference (in milliseconds) between the latest and the earliest time. Supports Timestamp only. An optional COLUMNS statement can be included to indicate additional input columns that should be added to the results even though they are not used to form groups. Examples Grouping Expression Definition group by ll_sourceUser count(*) The result will have two columns, the ll_sourceUser and count of users per distinct value. group by ll_sourceUser columns The result will have 4 columns ll_sourceUser, number of users for each distinct value of source user, minimum value of sys_eventTime and maximum value of sys_eventTime. count(ll_sourceUser), min(sys_eventTime), max(sys_eventTime) group by ll_sourceUser columns Duration(sys_eventTime) The result will have 2 columns, the source user and the duration. SORT BY Statement SORT BY causes the result rows to be sorted according to the specified expressions. By default, results are sorted in ascending order. ::= "SORT BY" ( "," )* ; A SORT BY expression can be the name of a column. If two rows are equal according to the leftmost expression, they are compared according to the next expression and so on. If they are equal according to all specified expressions, they are returned in an implementation-dependent order. The following functions are supported: ● ASC: Sort results in the ascending order. This is the default order. ● DESC: Sort results in the descending order. Examples Sorting Expression Definition sort by sys_eventTime ASC The result will be sorted by time in ascending order. TIBCO LogLogic® Unity User's Guide 40 Sorting Expression Definition sort by ll_sourceUser, sys_eventTime The result will be sorted by ll_sourceUser in ascending order (default), in case ll_sourceUser is the same, sort by sys_eventTime in descending order. DESC LIMIT Statement LIMIT indicates the maximum number of results that should be returned by the query. ::= "LIMIT" ; If you do not specify a LIMIT clause in the query, the default limit will be used. The default limit is set to 10,000. Example Limits Expression Definition limit 100 Limits the result set to top 100 rows. Search Examples SQL Expression EQL Expression Definition select sys_eventTime, use sample | columns sys_body from sample sys_eventTime, sys_body | where sys_eventTime sys_eventTime in TUE:WED Displays results from the sample source configuration where the records have the timestamp between '2014-02-02' (TUE) and '2014-02-03' (WED). between '2014-02-02' and '2014-02-03' select * from sample use sample | where sys_body like "Authentication" and '%Authentication%' and sys_eventTime between sys_eventTime between '2014-02-03' and '2014-02-03' and '2014-02-03' Displays results from the sample source configuration with "Authentication" in the message body. '2014-02-03' select * from sample use sample | sys_body like where sys_body like '%logon%' | limit 10 | '%logon%' and sys_eventTime in -10y Demonstrates an example of a 'like' statement to display a limit of 10 results. sys_eventTime between '2014-02-03' and '2014-02-03' limit 10 select * from sample use sample | sys_eventKey where sys_eventKey REGEXP REGEXP '[a-z0-9|]*' | '[a-z0-9|]*' and sys_eventTime in -1y | sys_eventTime in -1y limit 10 Demonstrates an example of the REGEXP expression matching. limit 10 TIBCO LogLogic® Unity User's Guide 41 SQL Expression EQL Expression Definition select * from sample use sample |sys_eventTime where sys_eventTime between '2014-02-03' and between '2014-02-02' and '2014-02-03' | sort by '2014-02-03' order by sys_eventTime DESC Displays events sorted by time for records having timestamp for the specified dates in the descending order. sys_eventTime DESC select * from sample use sample |sys_eventTime where sys_eventTime between '2014-02-03' and between '2014-02-02' and '2014-02-03' | sort by '2014-02-03' order by sys_eventTime DESC | limit sys_eventTime DESC offset 100 Displays top 100 results for records sorted by time in the descending order. 0 limit 100 select sys_eventTime, use sample | sort by sys_body from sample sys_eventTime DESC | LIMIT where sys_eventTime 100 Display sorted first page of results for events ordered by time in descending order. between '2012-02-14 14:34:34' and '2012-03-14 12:00:00' ORDER BY sys_eventTime DESC OFFSET 0 LIMIT 100 Displays grouped results based on the source users. select ll_sourceUser, use sample | group by count(*) from sample ll_sourceUser columns where sys_eventTime ll_sourceUser, count(*) | between '2014-02-02' and sys_eventTime between '2014-02-03' group by '2014-02-02' and ll_sourceUser '2014-02-03' select ll_sourceUser, use sample | group by max(sys_eventTime), ll_sourceUser columns min(sys_eventTime), max(sys_eventTime), count(*) from sample min(sys_eventTime), where sys_eventTime count(*) | sys_eventTime between '2014-02-02' and in -10y Displays the count of rows for distinct source users and its corresponding maximum timestamp and minimum timestamp. '2014-02-03' group by ll_sourceUser Event Correlation Language Reference LogLogic Unity Event Correlation Language (ECL) is very useful to find patterns in a given set of log. ECL is able to describe searches that are little complex for the regular EQL, especially when there is a need to join several types of events. Rules described in ECL can be used for advanced forensics searches and also for real-time alerting. TIBCO LogLogic® Unity User's Guide 42 Rule Structure A rule describes a pattern to look for within a given time window. It contains a list of event group definitions (at least one), and the correlation criteria that are used to join those event groups (if there is more than one event group). A rule can also be valid for only a given period of time. All mandatory parameters are explained below. The optional parameters are in square brackets [ ]. Valid From yyyy-MM-dd hh:mm:ss To yyyy-MM-dd hh:mm:ss ) ] [ ] USE (, )* Within [ d |h | m | s ][Fixed | Sliding ] … [Correlation … ] [Autofill] (Set AS )* [Inject Correlation Event] [ LIMIT CORRELATION EVENTS ] Each ruleset can have multiple rules. Each rule name must be unique in a defined ruleset. Parameter Description Rule The rule name defined using an identifier and the environment. For details, see Identifier Environment. USE The list of log sources used by the rule. Multiple log sources must be separated by comma (,). Within The time period is defined as an integer in days, hours, minutes, or seconds. Event Group Each event group describes the criteria that must combine events to be group together as part of the rule. This is equivalent to a single search in EQL. For details, see Event Group Structure. Correlation The correlation criteria describes the joins and other constraints that various event groups must meet to trigger a rule. For details, see Correlation Criteria. LIMIT Limit on number of correlation events is only effective for "replay" instances when INJECT CORRELATION EVENT is not set. The default limit is 10,000. TIBCO LogLogic® Unity User's Guide 43 Identifier Environment An identifier environment allows to specify the default tenant/domain/source configuration, when those parts are not present in a key identifier. The identifier environment is composed of: ● [Default Tenant ] ● [Default Domain ] ● [Default Source ] The identifier environment follows a hierarchical structure when resolving missing part in a identifier. The order is as follows: ● Event Group Environment ● Correlation Rule Environment ● Ruleset Environment ● Root Environment (defined outside ECL itself) — For Correlation REST API : this is the environment parameter — For Web application: this is related to the currently logged in user Limitations: ● The only possible value for tenant is: tenant1 ● The only possible value for domain is: shared ● The only possible value for source is: correlation Simple Identifier The simple identifier must be defined in letters, numbers, underscore (_), and dollar sign ($) with or without single quotes (' '). If not used single quotes (' '), use square brackets ([ ]), or back quotes (` `) For example: ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'$')* Key Identifier An identifier is composed of 4 parts separated by dots. An identifier part follows the syntax of the simple identifier. The identifier parts are : ● Tenant name ● Domain name ● Source configuration name ● Field name (or column name) The key identifier can be defined as shown below: [[[.].].] The is mandatory part. If the other identifier parts are not defined, they will be automatically extracted from the identifier environment. TIBCO LogLogic® Unity User's Guide 44 Limitations: ● The only possible value for tenant is: tenant1 ● The only possible value for domain is: shared ● The only possible value for source is: correlation Event Group An event group describes the criteria events should meet to be part of a rule. Event groups can be of the following 3 types : ● Required : the rule cannot be triggered if no event(s) matches this event group. This is the default type. ● Excluded : the rule will NOT be triggered if event(s) matches this event group. ● Optional : If events are matching this event group, they will be part of the triggering rule if other criteria are met. An event group may have : ● conditions on the number of events ● a filtering clause ● a grouping clause ● a set of having clause ● upper limits on the number of groups and events that may be created while this rule is run. This is a safeguard against a memory overflow. The event group can be defined as below: Event Group [ Is ( Required | Optional | Excluded ) ] [ With Delayed Evaluation ] [ At Least Events ] [ At Most Events ] [ ] [ Where ] [ With The Same [ As ] ( , [ As ] )* ] ( Having )* [ Limits Groups And Events ] When At Least parameter is defined, it requires at least an integer more than 0. If it is omitted, this implies at least 1. If Where clause is defined, it should match the expressions. It is evaluated as a Boolean. For details, see Expressions. Default Limits are 10000 groups and 100000 events. Expressions Expressions can be used to express how to compute a value in many situations. The different situations can be: ● in a condition ● in a grouping definition ● in field assignment TIBCO LogLogic® Unity User's Guide 45 [ ( + | - ) ] [ ( + | - ) ] "" { ( d | t | ts ) yyyy-MM-dd hh:mm:ss } True False Null $() ( ) *** / % + - Is [ Not ] Null Exists [ Not ] [ Any | All ] Like [ Not ] [ Any | All ] Regexp [ Any | All ] = [ Any | All ] != [ Any | All ] > [ Any | All ] >= [ Any | All ] <= [ Any | All ] < [ Any | All ] <> [ Any | All ] In ( , expression, … ) In / [ Any | All ] Between And Case ( When Then ] ( [ ] , [ ] , … ) The following operators are supported: ● Equals (=) ● Not equals (!=), (<>) ● Lower than (<) ● Lower or equal (<=) ● Greater than (>) ● Greater or equal (>=) ● ~= : This matches with the regular expression. ● In: — : Checks if value matches any one of the values in a set. Supports all data types. — /: Checks whether an IP address matches a network, defined as a network IP address and a network bitmask length. ● Between: Supports Timestamps, Long, and Integers. ● AND ● Functions ● Aggregation Functions ● Identifier Environment TIBCO LogLogic® Unity User's Guide 46 Examples: ( sys_eventType = “1234”) and ( sys_body like “%login failed%”) ( sys_bodySize > 30) and (sys_bodySize < 20) ( ll_eventID != null) and ( ll_eventID > -1 ) Functions Functions are used to compute a value as output from parameters as input. Some functions are predefined in the language. It is also possible to call a static java function provided by the user. Pre-defined Functions The functions which are available in ECL are listed below. Function Name Arguments Returns (String) Length of string 1. lower (String) Lower case of string 1. upper (String) Upper case of string 1. trim (String) Trimmed string 1 (without leading and trailing spaces). substitute (String 1, String 2, String 3) Substitute string 2 by string 3 in string 1. left (String, Int) left characters of string 1. right (String, Int) right characters of string 1. mid (String, Int 1, Int 2) Characters from string1 starting at offset for a length of . Findposition (String 1, String 2) Index of the first occurrence of string2 within string1, -1 if no occurrence is found. concatenate (String 1, String 2, …) Concatenation of all strings passed as arguments. String functions len char_length character_length length substr substring List functions TIBCO LogLogic® Unity User's Guide 47 Function Name Arguments Returns size List Size of the list. Condition, then, else Returns the Then value if condition is true, otherwise it should return the Else value. Conditional functions IIF IIF(true, “a”, “b”) returns “a” IIF(false, ”a”,”b”) returns “b” Smart List functions lookup (String 1, String 2) The value associated with String2 in the smart list named String1. isInList (String 1, String 2) True if the value String2 is defined in smart list named String1. (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) The expression, which should evaluate to a string, is interpreted as a time according to the supplied formatString. If the conversion fails, null is returned, unless a default string is provided, which is interpreted as a time and returned. Conversion functions ToTimestamp If timezone is omitted or is empty, the system default timezone is used. ToIP (expression_ or (expression, defaultValue) Convert the expression to an IP address (Java InetAddress).If the conversion fails, null is returned, unless a default string is provided, which is interpreted as an IP address and returned. TIBCO LogLogic® Unity User's Guide 48 Function Name Arguments Returns ToTimestampString (expression, formatString) or (expression, formatString, timezone) or (expression, formatString, timezone, defaultValue) Same as ToTimestamp, except the conversion is in the opposite direction to get a printable timestamp. ToInt (expression) or (expression, defaultValue) The obvious conversion to integer with default value taken if not convertible. ToLong (expression) or (expression, defaultValue) The obvious conversion to Long with default value taken if not convertible. ToString (expression) or (expression, defaultValue) The obvious conversion to String with default value taken if not convertible. ToFloat (expression) or (expression, defaultValue) The obvious conversion to Float with default value taken if not convertible. ToBool (expression) or (expression, defaultValue) The obvious conversion to Boolean with default value taken if not convertible. ToDouble (expression) or (expression, defaultValue) The obvious conversion to Double with default value taken if not convertible. If timezone is omitted or is empty, the system default timezone is used. Aggregation Functions For expressions used in the Having clause must contain at least one aggregation function. Count Count Sum ( Avg ( Max ( Min ( Var ( Stdev ( ( [ [ [ [ [ ( * ) [ Distinct Distinct | Distinct | Distinct | Distinct | Distinct | [ Distinct | All All ] All ] All ] All ] All ] | All ] Limit ) Limit ) Limit ) Limit ) Limit ) Limit ) ] Limit ) Option Definition (*) This will apply the function for any event with no additional constraints. All This will apply the function on all values that are not null. TIBCO LogLogic® Unity User's Guide 49 Option Definition Distinct This will only apply the function once per distinct values. Sum This is the total value. Avg This is the average value. Max This is the maximum value. Min This is the minimum value. Var This is the variance. Stdev This is the standard deviation function. Having Clause The Having clause add additional constraints on the events which have passed the filter and are grouped by the rule. At (Least | Most) Distinct As Limit Count Of Being (Greater | Less) Than Percentage Of Being (Greater | Less) Than % The Having clause expression must contain at least 1 aggregation function. The supported parameters are: ● Count Of: count the number of time two expressions are equals and check that this value is greater/ less than a boundary. ● Percentage Of: count the number of time two expressions are equals and make a ratio of this count versus the number of events in the group, then check whether the value is below/above a value expressed as percent. The Having clause may also be an expression using aggregation functions and resolving to a boolean. Correlation Criteria Correlation criteria can be of the following 3 types: ● a join condition describing which fields should be equals in two event groups ● a sequencing constraint that describes the relative order in which two event groups should occurs ● an expression criteria that describes a condition among fields of different event groups > == > (Begins | Ends) [At Least [ d | h | m | s ]] [Up To [ d |h | m | s ]] (Before | After) (Begins|Ends) This is an expression criteria that is used to describe a condition between fields that belongs to different event groups. fieldIdentifier for keys> For example, group1->sum_bytes >= group2->sum_bytes TIBCO LogLogic® Unity User's Guide 50 The fields referenced in a join must be grouping fields for their respective event groups. Correlation Blok (ECL) Examples When a Blok triggers, it creates a correlation event result in forensic/search mode and executes the actions of the associated trigger (create alert, notify by email) in the realtime mode. Blok use sample Within 30m Event Group [My Events] use sample Within 30m Event Group [My Events] Having at least 1 distinct [ll_sourceDomain] Having at least 1 distinct [ll_type] Definition Blok Example 1: This Blok will trigger a new alert at the first event and will accumulate all events during 30 minutes time period. Blok Example 2: This Blok will do the same as Blok Example 1 but the alerts generated will then give information about the number of distinct ll_sourceDomain / ll_type and their values. use sample Within 30m Event Group [My Events] where [ll_type] ="Network" Having at least 2 distinct [ll_sourceIP] Blok Example 3: use sample Within 30m Event Group [suspiciousSources] At least 100 events where [ll_type] ="Network" With the same [ll_sourceIP] Having at least 1 distinct [ll_eventStatus] Blok Example 4: use sample Within 30m Event Group [suspiciousUsers] At least 100 events where [ll_type] ="Network" With the same [ll_sourceUser] Having at most 1 distinct [ll_eventStatus] Having at least 10 distinct [ll_sourceIP] Blok Example 5: This Blok will filter events which have ll_type equal to "Network", and at least 2 distinct values of ll_sourceIP. This Blok is looking for at least 100 events with the same criteria as the previous one, coming from the same ll_sourceIP and giving information about the number of distinct ll_eventStatus and their value. This Blok filters the event the same way as the previous one, and is looking for 100 events from the same ll_sourceUser that have at least 10 distinct ll_sourceIP and at most 1 distinct ll_eventStatus. TIBCO LogLogic® Unity User's Guide 51 Blok Definition use sample Within 30m Event Group [successAudit] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Success Audit" With the same [ll_sourceUser],[ll_sourceIP] Event Group [failure] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Failure" With the same [ll_sourceUser],[ll_sourceIP] Correlation successAudit->[ll_sourceIP]==failure>[ll_sourceIP] successAudit->[ll_sourceUser]==failure>[ll_sourceUser] Blok Example 6: use sample Within 30m Event Group [successAudit] is excluded at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Success Audit" With the same [ll_sourceUser],[ll_sourceIP] Event Group [failure] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Failure" With the same [ll_sourceUser],[ll_sourceIP] Correlation successAudit->[ll_sourceIP]==failure>[ll_sourceIP] successAudit->[ll_sourceUser]==failure>[ll_sourceUser] Blok Example 7: use sample Within 30m Event Group [users] where [ll_eventStatus] ="Failure" OR [ll_eventStatus] ="Success Audit" With the same [ll_sourceUser] Having at least 2 distinct [ll_eventStatus] Blok Example 8: use sample Within 30m Event Group [users] where [ll_eventStatus] ="Failure" OR [ll_eventStatus] ="Success Audit" With the same [ll_sourceUser] Having at least 2 distinct [ll_eventStatus] Having sum([ll_eventStatus] ="Failure") > ( 2 * sum([ll_eventStatus] ="Success Audit")) Blok Example 9: This Blok looks at two groups of event happening within 30 minutes. The first event group is success audit from the same ll_sourceIP/ll_sourceUser and the second group is failure status grouped the same way. The Blok will be triggered if the fields grouped on both event groups are same. Same as the previous Blok but this time the Blok will trigger if there is only failure events within 30m for the same ll_sourceIP / ll_sourceUser. This Blok looks for users that have events with ll_eventStatus equal to either failure or success. Same as the previous Blok with an additional constraints that there are twice as many failures than success events. TIBCO LogLogic® Unity User's Guide 52 About Bloks To analyze your data faster, LogLogic Unity allows you to create different types of Bloks to help you accelerate your search process. A Blok is a contextual element or filter that fits with other elements to form a search query. Build and save different Bloks that can be used in future searches rather than searching every time with the same filter. Bloks are reusable elements of a query. You can combine any types of Bloks together, except the correlation Blok, to create complex queries. LogLogic Unity supports the following types of Bloks: ● Filter Bloks: contain filter statements. ● Correlation Bloks: contain different correlation rules. ● Time Bloks: contain absolute and relative time ranges. You can have one or more filters in a Blok. If you realize that you need to add another Blok to the existing Blok, you can add more filters and build another Blok. Only one correlation Blok can be used at a time in a query. You can add new Bloks and modify existing Bloks from the Search tab. Similarly, you can manage all types of Bloks using the Admin > Bloks menu. For detailed information, see Manage Bloks. When entering a Blok name in the Search field, you need to start with the prefix defined for each type of Blok as listed below. Content assist can help you by showing all possible values for that type of Blok. ● time.Blok name ● filter.Blok name ● correlation.Blok name For example, create a Blok and use it in a search query: ● create and save a filter Blok that has user='joe' AND body like '%security%'. Now when you run a query using this Blok, only events with "joe and security" will be retrieved. ● use this filter Blok and add another element or filter to it, for example, type user='John' to the same query to create a more complex query. For example, filter Blok AND user='John'. Now when you run a query using this Blok, events with "joe and security and john" will be retrieved. About Filter Bloks You can create filter Bloks that contain one or multiple filters. Each filter comprises of one or multiple terms. A filter Blok supports valid EQL or SQL statements. You can have either one or multiple filters in a Blok. If you realize that you need to add another Blok to the existing Blok, you can add more filters and build another Blok. Multiple Bloks of different types, except the Correlation Blok, can be used in a single search query. For detailed information on how to create a Blok, see Adding a Blok. When entering the Blok name in the Search field, start with the prefix filter for any existing filter Blok. Content assist can help you by showing all possible values for that type of Blok. For detailed information about valid filters, see Filter Statement. TIBCO LogLogic® Unity User's Guide 53 About Correlation Bloks For your forensic needs, search with a created correlation Blok or Event Correlation Language (ECL) rule on historical (past) data. You can create correlation rules in order to use them for alerts or searches. You can manage triggers using correlation Bloks so that you can receive alerts on real-time data. A Correlation Blok is a correlation rule without its header (rulename, description, author, and date). For detailed information on how to define correlation rules, see Event Correlation Language Reference. You cannot combine a correlation Blok with other Blok types in a single query. Only one correlation Blok can be used at a time in a query. For detailed information on how to create a Blok, see Adding a Blok. When entering a Blok name in the Search field, start with the prefix correlation for any existing correlation Blok. Content assist can help you by showing all possible values for that type of Blok. The correlation search results are displayed every time the rule's conditions are met. For more information, see Correlation Format. Viewing All Bloks The default or existing Bloks can be easily used to quickly search your data. The default Bloks have preset values. You cannot modify or delete the default Bloks. However, you can update or delete any custom Bloks. Procedure 1. From the Search tab, click located next to the Search field and select Choose Blok. 2. Select the type of Blok from the list. The options are All, filter, correlation, and time Bloks. 3. In the Find field, type the Blok name to quickly find the desired Blok. 4. Select the Blok name from the list of Bloks. The Description and Source statement fields are auto-populated based on the selected Blok. 5. Click Save to add the Blok in the Search field. If you select a time Blok, it is displayed in the Time field. 6. Click to view results for the defined Blok. Adding a Blok If you usually search for events that provide you with specific information such as user name or severity, you can create a custom Blok for that criteria and save it for later use. Procedure 1. From the Search tab, click located next to the Search field and select New Blok. 2. Select the Type of Blok from the list. The options are Filter and Correlation Bloks. 3. Enter the name of the Blok in the Name field. It must be a unique name that consists of a single word with no special characters. This is a mandatory field. 4. Enter the description of the Blok in the Description field. TIBCO LogLogic® Unity User's Guide 54 5. Enter the statement of the source in the Source statement field. Make sure to enter a valid syntax. Filter and Time Bloks support EQL and SQL syntax. Correlation Bloks support ECL syntax. For syntax information, see Search Syntax Reference. 6. Click Validate to verify the statement. Click Format Statement to format the statement. The Validate option is available only for correlation Bloks. 7. Click Save to save the new Blok. The new Blok is added in the Choose Blok list and displayed in the Search field. Modifying Bloks You can modify the user-defined custom Bloks at any time. You cannot modify default Bloks. Procedure 1. From the Search tab, update the statement in the Search field. Content assist shows you contextual matches and completions for each keyword as you type into the Search field. For syntax information, see Search Syntax Reference. 2. Click located next to the Search field and select Save as Blok. 3. Update the information. For information about each field, see Adding a Blok. If you update the ECL rule in a correlation Blok, make sure to deploy the related triggers for the updated rule to take effect. For details, see Manage Triggers. 4. Click Save to save as a new Blok. The new Blok is added in the Choose Blok list and displayed in the Search field. Deleting Bloks You can delete the user-defined custom Bloks at any time. You cannot delete default Bloks. Once the Blok is deleted, active queries are not affected, but you cannot start a new query with a deleted Blok. Queries in the Search History that use this Blok cannot be started again. When a trigger is active, you cannot delete the associated correlation Blok. Procedure 1. Click the Admin menu and select Bloks . 2. Click the check box located next to the Blok name that you want to delete and click You can select one or multiple Bloks. . 3. In the confirmation window, click Ok to delete the selected Blok. The Blok management page is updated immediately. About Time Bloks Analyzing events based on a certain time range can help correlate results and find the root cause faster. You can narrow your search results to a specific time range using the Time Blok. You can use the preset time Blok or create your custom time Blok that you can use any time. Each time Blok is translated in a statement before executing the query. When entering the time Blok name in the Search field, start with the prefix time for any existing time Blok. Content assist can help you by showing all possible values for that type of Blok. For detailed information on how to create a time Blok, see Adding a Time Blok. TIBCO LogLogic® Unity User's Guide 55 By default, the time range is set to last hour. You can define the absolute or relative time. For valid time ranges, see Time Range Expressions. Viewing All Time Bloks The default or existing time Bloks can be easily used to quickly search your data. The default time Bloks have preset time ranges. You cannot modify or delete the default time Bloks. However, you can update or delete user-defined time Bloks. Procedure 1. From the Search tab, click located next to the Time field and select Choose Blok. 2. In the Find field, type the Blok name to quickly find the desired time Blok. 3. Select the Blok name from the list of Bloks. The Description and Source statement fields will be auto-populated based on the selected Blok. 4. Click Save to add the Blok in the Time field. The selected time Blok is displayed in the Time field. 5. Click to view results for the defined time range. Adding a Time Blok If you usually search for events that are in the specific time range, you can create a custom time Blok for that time range and save it for later use. Procedure 1. From the Search tab, click to open a calendar window. located next to the Time field and select the Select a date range link 2. Specify the date using the From and To fields. Similarly, specify the time in Hours and Minutes and click Add range. The selected date and time range is displayed in the Time field. 3. Alternatively, type in the time expression in the Time field. Content Assist shows you typeahead or contextual matches and completions for each keyword as you type it into the search field. To define a valid time statement, see Time Range Expressions. 4. To save a new time Block, click next to the Time field and select Save as Blok. Alternatively, to add a new Blok, select New Blok. 5. In the Add new Blok window, enter the information in the following fields: a) Name - Enter the name of the Blok. It must be a unique name that consists of a single word with no special characters. This is a mandatory field. b) Description - Enter the description of the Blok. c) Source Statement - The statement of the source (time expression). 6. Click Save to save the new time Blok. The new time Blok is added in the Choose Blok list. TIBCO LogLogic® Unity User's Guide 56 Modifying Time Bloks You can modify the custom time Bloks at any time. You cannot modify default time Bloks. Procedure 1. From the Search tab, update the time range expression in the Time field. For detailed information about valid time statements, see Time Range Expressions. 2. To save a new time Block or update the existing Blok, click Save as Blok. next to the Time field and select 3. Update the information. For information about each field, see Adding a Time Blok. 4. Click Save to save the new time Blok. The new time Blok is added in the Choose Blok list. Manage Bloks A Blok is a contextual element or filter that fits with other elements to form a search query. Build and save different Bloks that can be used in future searches rather than searching every time with the same filter. For detailed information on how to search using Bloks, see About Bloks. You can manage all types of Bloks using the Blok management page. Click the Admin menu following: ● and select Bloks . From the Blok management page, you can do the Find Bloks You can quickly find the desired Blok by typing the Blok name in the Find field. As you start typing the Blok name in the Find field, the Blok management page is automatically refreshed showing your selection. ● View Bloks based on filters You can use filters to easily find Bloks. Click the View list to viewall Bloks in the system. ● Sort Bloks You can sort any column in the ascending or descending order. Click on the column name or click the arrow (that appears on the right side of the column name when you click in the column) to sort the column. ● Add a new Blok Click ● to add a new Blok. For instructions, see Adding a Blok. Edit existing Bloks Select the Blok name that you want to update. The Details panel appears on the right side of the page. Click the Edit link to update. For instructions, see Modifying Bloks. TIBCO LogLogic® Unity User's Guide 57 ● Delete Bloks You can delete single or multiple Bloks. For instructions, see Deleting Bloks. ● Show or hide columns You can show or hide columns, except the mandatory column, from the table . Click to view all available columns in the table. Select the check box to show the column. Clear the check box to hide the column from the table. The Blok management page is updated immediately. The Bloks information is described below: Column Description Name The name of the Blok. Description The description of the Blok. Type The type of Blok. Created by The user who created the Blok. TIBCO LogLogic® Unity User's Guide 58 Manage Triggers Triggers describe what action should be taken once a correlation Blok is triggered. If there are several triggers associated with the same correlation Blok, they all will be triggered. Triggers can be compressed using the maximum number of triggers per time period setting. You can enable and disable triggers at any time but they must be synchronized in order to be activated on the correlation node. An alert is an email notification that gets sent out when a trigger is activated. You can set multiple email notifications for a single alert. For more information about alerts, see Monitor Alerts. You cannot synchronize only one trigger. The synchronization process takes all enabled and disabled triggers in the system and deploys them to the correlation node. Viewing Triggers You can view all defined triggers, add new triggers, edit triggers, enable and disable triggers, and delete triggers. Click the Admin menu ● and select Triggers . From the Triggers page, you can do the following: Filter triggers You can quickly find the desired trigger by typing the trigger name in the Find field. As you start typing a trigger name in the Find field, the Triggers page is automatically refreshed showing your selection. ● View triggers based on filters You can use filters to easily find triggers. Click the View list to view different filters. ● All - view all triggers in the system. ● Custom - view based on user-defined filters. For example, High Severity, Enabled, and Disabled. The Triggers information is described below: Column Description Enabled Indicates if the trigger is enabled or disabled. ● ON indicates enabled. ● OFF indicates disabled. Name The name of the trigger. Status The status of the trigger. The status options are: ● Active ● Inactive ● Pending TIBCO LogLogic® Unity User's Guide 59 Column Description Severity The severity of the trigger. The default options are: ● Info ● Low ● Medium ● High An Admin (a user with Administrator privileges) can configure severity options. The options may differ if they have been configured. SLA The Service Level Agreement (SLA) time is the time by which an operator is expected to acknowledge the generated alert. Description The description of the trigger. Type The type of trigger; an Alert. Category The category of the trigger. The default options are: ● Attack on third party ● Authorized Activity ● Authorized security testing ● Emergency changes ● False positive ● Known error ● LogLogic Event ● Network Noise ● Security Alert ● Suspicious Activity ● Unauthorized Activity ● Unknown An Admin (a user with Administrator privileges) can configure category options. The options may differ if they have been configured. Adding a Trigger You can add a new trigger that can activate an alert. Define the correlation rule associated with the alert and who should receive alert emails when the trigger is activated. You can also set the alert notifications. Procedure 1. Click the Admin menu and select Triggers . TIBCO LogLogic® Unity User's Guide 60 2. From the Triggers page, click to add a new trigger. 3. In the Trigger details section, enter the following information: a) Created by - The name of the creator is auto-populated and cannot be modified. b) Trigger name - The name of the trigger. c) Description - The description of the trigger. d) Severity - Select the severity of the trigger from the list. e) Category - Select the category of the trigger from the list. An Admin (a user with Administrator privileges) can configure Severity and Category options. 4. Select the Correlation Blok that you want to use from the list. You must specify a correlation rule that defines how to gather events into alerts according to alert's value fields. For information on how to define correlation rules, see Event Correlation Language Reference. 5. If you want to add a new correlation Blok, click and select New Blok. For information about how to add a new Blok, refer to Adding a Blok. 6. In the Notifications section, enter the following information: a) Type - The email notification. b) c) d) e) Make sure to configure SMTP connection. For instructions, see Configuring SMTP Connection. To - The email address of the person who should receive an alert email. Using the comma (,) separator, you can add multiple recipients. CC - The email address of the person who should receive a copy of an alert email. Using the comma (,) separator, you can add multiple recipients. Subject - The subject of an email. Message - The description of the alert. You can use the defined variables from the list on the right side. Double-click on the variable to add it into the Message field. The variables may change based on your data. Click to add a new notification. You can add multiple notifications for a single trigger. 7. In the Configure notifications section, enter the following information: a) Enable - Click the slider to ON setting to enable the trigger. Click the slider to OFF setting to disable the trigger. b) Maximum alerts per time period - Enter the maximum number of alerts to be triggered for a specified time period. Specify the time period from the list. 8. Click Save to add a new trigger. The newly added trigger is displayed on the Triggers page. TIBCO LogLogic® Unity User's Guide 61 Configuring SMTP Connection For triggers to send alert notifications, a valid SMTP configuration is required. Procedure 1. Create a configuration template as shown below. { "configurations": [ { "smtp": [{ "description": "smtp-1", "hostname": "smtp.gmail.com", "port": 465, "security": "ssl", "username": "", "password": "", "fromAddr": "" }] } ] } 2. Save the file to your local drive. 3. Enter the username and password information. 4. Enter the email address in the fromAddr field. The alert notifications will be sent from this email address. 5. Run the following command to upload the configuration file: llconf -c -f Editing Triggers You can update existing triggers. Procedure 1. Click the Admin menu and select Triggers . 2. From the Triggers page, select the trigger row that you want to update. The Details panel appears on the right side of the page. 3. Click the appropriate Edit link to update that section. For information about each section, see Adding a Trigger. 4. Click Save to save the updated information. The updated trigger is displayed on the Triggers page. What to do next Whenever you update any trigger in the system, you must synchronize and deploy all triggers. Synchronizing Triggers The synchronization process takes all enabled and disabled triggers and deploys them to the correlation node. This process resets all triggers in the system. Procedure 1. Click the Admin menu and select Triggers . TIBCO LogLogic® Unity User's Guide 62 2. From the Triggers page, update the trigger information. For details, see Editing Trigger. When the triggers are updated, the number of updates are displayed on the Sync The Sync button. button is enabled only when there are any updates to the existing triggers. 3. Click to reset all triggers. A confirmation window appears showing all triggers in the system that will be reset. 4. Click Sync to reset all triggers and deploy them to the correlation node. Once all triggers are reset, the Sync button is disabled on the Triggers page. Enabling/Disabling Triggers Triggers can be enabled or disabled and must be synchronized in order to be activated on the correlation node. Procedure 1. Click the Admin menu and select Triggers . 2. To enable the trigger, click the slider in the Enable column to ON. 3. To disable the trigger, click the slider in the Enable column to OFF. What to do next Once you update the trigger, click the Sync button to reset and deploy all triggers. Deleting Triggers You can only delete disabled triggers. Procedure 1. Click the Admin menu and select Triggers . 2. From the Triggers page, select the trigger that you want to delete by selecting the check box located next to the Enable column and click . The Delete button is available only for disabled triggers. If the trigger is enabled, click the slider to OFF to disable the trigger and then you can delete the trigger. 3. In the confirmation window, click Delete to delete the trigger from the system. The Triggers page is updated immediately. TIBCO LogLogic® Unity User's Guide 63 Monitor Alerts An alert is generated when real-time events are matching a correlation Blok that has an active trigger linked to it. An alert can also be distributed by an email to a pre-defined list of people. By default, the retention period is 90 days for all generated alerts. For email notifications, ensure to configure SMTP connection. For instructions, see Configuring SMTP Connection. For information on how to define triggers, see Manage Triggers. Viewing Alerts You can view all triggered alerts, view acknowledged alerts, and filter on the existing alerts in the system. From the Alerts tab, you can do the following: ● Filter alerts You can quickly find the desired alert by typing the alert name in the Find field. As you start typing the alert name in the Filter field, the Alerts tab is automatically refreshed showing your selection. ● View alerts based on filters You can use filters to easily find alerts. Click the View list to view different filters. ● ● All - view all alerts in the system. ● Custom - view based on user-defined filters. For example, High Severity, Acknowledged, and Unacknowledged. Acknowledge alerts Acknowledging an alert indicates that you have recognized the alert. Once you acknowledge the alert, your user name gets associated with that alert. For instructions on how to acknowledge alerts, see Acknowledging Alerts. ● Auto-refresh Alerts table Click the down arrow next to the refresh button to set the refresh interval in seconds. Enter the time in seconds. The Alerts table is refreshed as per the defined time interval. By default, it is refreshed every 30 seconds. ● Sort alerts You can sort any column in the ascending or descending order. Click on the column name or click the arrow (that appears on the right side of the column name when you click in that column) to sort the column. ● Show or hide columns You can show or hide columns, except the mandatory column, from the table . Click to view all available columns in the table. Select the check box to show the column. Clear the check box to hide the column from the table. The Alerts tab is updated immediately. The Alerts information is described below: TIBCO LogLogic® Unity User's Guide 64 Column Description Severity The severity of the trigger. The default options are: ● Info ● Low ● Medium ● High An Admin (a user with Administrator privileges) can configure severity options. The options may differ if they have been configured. SLA Expiration The Service Level Agreement (SLA) expiration time is the time by which an operator is expected to acknowledge the alert. When the SLA time expires, it displays the time in negative hours/days in the field. Status The icon indicates if the alert is active or expired. Acknowledged ● The indicates that the alert is expired. ● The indicates that the alert is acknowledged. The indicates that the alert is acknowledged. Name The trigger name associated with the trigger. Description The description of the alert. Category The category of the trigger. The default options are: ● Attack on third party ● Authorized Activity ● Authorized security testing ● Emergency changes ● False positive ● Known error ● LogLogic Event ● Network Noise ● Security Alert ● Suspicious Activity ● Unauthorized Activity ● Unknown An Admin (a user with Administrator privileges) can configure category options. The options may differ if they have been configured. TIBCO LogLogic® Unity User's Guide 65 Column Description Notified The date and time of the alert notification. Elapsed time The time since the alert was created. Last updated The time when the alert is last updated. Acknowledging Alerts Acknowledging an alert indicates that you have received and recognized the alert. Once you acknowledge the alert, your name gets associated with that alert. Procedure 1. From the Alerts tab for a single or multiple alerts, select the check box located on the left side of the alert. For all alerts in the system, select the check box located on the left side on the top of the table. 2. Click Acknowledge to acknowledge selected alerts. 3. In the Acknowledge Alerts window, the name of a person who acknowledges the alert is autogenerated in the By field. 4. Select the alert Severity from the list. 5. Select the alert Category from the list. 6. Enter any comments in the Comment field. 7. Click Acknowledge to acknowledge alerts. The Alerts table shows icon in the Status column and a checkmark column for all acknowledged alerts. in the Acknowledged TIBCO LogLogic® Unity User's Guide 66 Viewing Alert Details You can view details of any generated alert. Procedure 1. From the Alerts tab, click the alert Name to view the alert details. You can view alert details, history, associated correlation rule, and event group details. 2. Click on the heading or click located on the right side to display the details for that section. Click to hide the details. 3. To acknowledge the alert, click the Acknowledge link. For details, see Acknowledging Alerts. TIBCO LogLogic® Unity User's Guide 67 Viewing Event Group Details Each event group describes the criteria that must combine events to be grouped together as part of the correlation rule. This is equivalent to a single search query defined in EQL. Procedure 1. From the Alerts tab, click the alert Name to view the alert details. The alert details window is displayed. You can view alert details, history, associated correlation rule, and event group details. Click section. Click located on the right side to display the details for that to hide the details. 2. Click the event group count link (as shown in the above illustration) to view the associated event count query. A new search tab is added showing the event count query in the Search field. The Result tab displays the retrieved results in the Charts, Columns, and Data panels. The following illustration displays a new Search tab 2 > My > details opened with the event count query added in the Search field and retrieved results in the Charts, Columns, and Data panels. TIBCO LogLogic® Unity User's Guide 68 Manage Source Configurations LogLogic Unity parses log data into a structured format to enhance search and analysis. Based on the log source type, you can define how to parse your data and which columns to extract. You can create a source filter that defines which log source to use for parsing based on the data relevance. For multiple log sources, the order of precedence can be defined in a specified query. The system columns will be extracted from event metadata in a tabular format. All system columns are displayed with the prefix sys_ and all columns from built-in parsers are displayed with the prefix ll_ in the Columns panel. LogLogic Unity provides built-in source configurations. For a detailed list, see Supported Log Sources. LogLogic Unity supports two types of parsers: ● Key-value Parser – This parser uses simple key-value pair parsing rules to extract keys and values. The parser recognizes patterns like k1=v1, k2=v2, k3=v3. You can use key-value pair separators, for example, space, comma (,), or semi-colon (;), and key and value separators, for example, equal sign (=) or colon (:). ● Columnar Parser – The data is extracted into different columns. This parser operates on data that is separated by a character or a sequence of characters, such as a comma, or a tab. There is no keyvalue, just the value. The data from different log sources will extract different columns depending on keys identified in the data. When referring to a column in a column expression, it is referred to as $. So the first column is referred to as $1, the second column will be $2 and so on. Viewing Source Configurations You can view all defined source configurations, add new configurations, edit configurations, enable and disable configurations, and delete configurations. Click the Admin menu you can do the following: ● and select Source configuration . From the Source configuration page, Filter configurations You can quickly find the desired configuration by typing the rule name in the Find field. As you start typing a rule name in the Find field, the Source configuration page is automatically refreshed showing your selection. ● View configurations based on filters You can use filters to easily find configurations. Click the View list to view different filters. ● ● All - view all configurations in the system. ● Custom - view based on user-defined filters. For example, Created by Me, and Created by System. Sort configurations You can sort any column in the ascending or descending order on the Source configuration page. Click on the column name or click the arrow (that appears on the right side of the column name when you click in that column) to sort the column. ● Show or hide columns to view all You can show or hide columns, except the mandatory column, from the table . Click available columns in the table. Select the check box to show the column. Clear the check box to hide the column from the table. The Source configuration page is updated immediately. The Source configuration information is described below: TIBCO LogLogic® Unity User's Guide 69 Column Description Enabled Indicates if the configuration is enabled or disabled. ● ON indicates enabled. ● OFF indicates disabled. All enabled configurations can be searched using the source filter on the Search tab. Name The name of the source configuration. Created by The name of the user who created the configuration. Date created The date when the configuration was first created. Last edited The date when the configuration was last updated. Adding a Source Configuration You can add a new source configuration that can be activated to analyze results in the normalized format. Procedure 1. Click the Admin menu and select Source configuration 2. From the Source Configuration page, click . to add a new configuration. 3. Enter the name of the rule in the Source name field. 4. Click the slider to ON to enable the configuration. Click the slider to OFF to disable the configuration. All enabled configurations can be searched using the source filter from the Search tab. 5. Enter the description in the Description field. 6. In the Enter a source filter field, define the source filter statement that is assigned for this parsing rule. The source filters bind multiple source configurations for parsing. Source filters can be used on system columns. All EQL source filter statements are supported. For details, see Filter Statement. For example, sys_sourceType=ID (device type ID that is retrieved from LMI) If you specify multiple source configurations, the first configuration whose filter matches an event will be used to parse that event, extracting the columns specified by that configuration. 7. Paste the sample log data in the Parse sample data field. This data can be helpful in defining the parsing configuration based on the log source. TIBCO LogLogic® Unity User's Guide 70 8. From the Parsing list, select the type of parser you want to use. The options are: Key value and Columnar. ● For the Key value parser, define the following information: — Values separator - Enter the delimiter that you want to use to separate key-value pairs. You can add only one separator at a time. The delimiters are case-sensitive. For example, user=bob,vm=windows where user=bob is one pair and vm=windows is another pair separated with delimiter comma (,). — Key/value separator - Enter the delimiter that you want to use to separate keys from their values. The delimiters are case-sensitive. For example, user=bob where user is a key and bob is a value separated with delimiter equal sign (=). To specify a , enter backslash s (\s) and for a , enter backslash t (\t). ● For the Columnar parser, define the following information: — Separator - Enter the delimiter that you want to use as a column separator. The separator can be a string of one or more characters, or a Java regular expression. The delimiters are case-sensitive. For example, bob,windows where comma (,) is a character used to separate two columns. — RegEx - Use this option to define how the separator should be interpreted. Select ON to use as a Java regular expression or OFF to use as a literal string. — Escape character - Define a character that is actually used to escape the character used as a column delimiter. The delimiters are case-sensitive. For example, if you use a comma as a column separator and your column value has a comma in it, that value would have to be escaped so that a parser does not think that the instance of the comma is the start of a new column. — Max columns - Enter the maximum number of columns to be extracted. If more columns than maxColumns are found, then the content of the additional columns will be included in the last column. For example, if the separator is and the maxColumns value is 3 for a message like “a b c d”, then we would have 3 columns with values “a”, “b” and “c d”. — Trim values - If defined ON, then the extra (white) space that is generated at the beginning and end of the column is removed. If defined OFF, the extra space is not removed. TIBCO LogLogic® Unity User's Guide 71 9. Once you define the parser type, all system columns are extracted in the Manage your column list panel. You can add and delete columns. Click to add a column. Hover over the row, the Delete button appears on the right side of the row that allows you to delete the column. If you want to autogenerate the column list after updating the parser separators, click Auto generate columns to extract columns based on the updated value. ● Name - The name of the column that should appear in the results. Click in the row to add or update any column name. The column name must start with a letter. ● Type - The data type of the column. Click in the column to add or update the supported data types. Select the data type from the list. ● Expression - Define how to map values extracted by parser into defined columns. You can use arithmetic operators and conversion functions when defining an expression. The conversion functions are typically use when you need to define new columns where the expressions for new columns can use conversion functions to convert between data types and combine them using various operators. For details about the arithmetic operators, see Filter Statement section and for conversion functions, see Predefined Functions. The type of expression depends on the parser type: — For Key value parser, the expression uses a key name to extract the value for the column. For example, user is the value of the key user in the log line or null if the key is not present. — For Columnar parser, the expression uses the $ identifier where n is the column number for the value of column n. For example, $2 is the value of the column 2. If you select the parser and the column list is empty, the parser will try to guess columns from the sample data. 10. Select the Include system columns check box to include system columns in the results. If the check box is not selected, only the user defined columns, sys_eventTime, and sys_body columns will be displayed in the results. For a list of system columns, see About Columns. 11. After modifying configuration settings, click table. to refresh the Preview table to view the updated 12. Click Save to add a new source configuration. The Source configuration page displays the newly added configuration. Enabling/Disabling Source Configurations Source configurations can be enabled or disabled at any time. All enabled configurations can be searched using the source filter on the Search tab. By default, the system source configuration is enabled. You cannot disable the system source configuration. Procedure 1. Click the Admin menu and select Source configuration . 2. To enable the source configuration, click the slider in the Enabled column to ON. 3. To disable the source configuration, click the slider in the Enabled column to OFF. TIBCO LogLogic® Unity User's Guide 72 Editing Source Configurations You can update existing configurations at any time. You cannot update the system source configuration and LogLogic Unity built-in source configurations from the system. See Supported Log Sources list for details. Procedure 1. Click the Admin menu and select Source configuration . 2. From the Source Configuration page, click the rule name that you want to update. The Details panel appears on the right side of the page. 3. Click the Edit link to update the configuration. For detailed information, see Adding a Source Configuration. 4. Click Save to save the updated information. The Source Configuration page is updated immediately. Deleting Source Configurations You can delete single or multiple configurations from the system. You cannot delete the system source configuration and LogLogic Unity built-in source configurations from the system. See Supported Log Sources list for details. Procedure 1. Click the Admin menu and select Source configuration . 2. From the Source Configuration page, click the check box located next to the rule name that you want to delete and click . 3. In the confirmation window, click Ok to delete the configuration from the system. The Source configuration page is updated immediately. TIBCO LogLogic® Unity User's Guide 73 Supported Log Sources LogLogic Unity supports message body text search for all of the log sources supported by LogLogic LMI and also supports advanced searching of source-specific parsed columns for the following sources via LogLogic Parser. For details, see the TIBCO LogLogic® Log Source Packages documentation. Note that not all event types supported by LogLogic LMI may be supported by LogLogic Unity. Log Source Versions/Platforms Device Category ADS - Microsoft Active Directory Service AD Service on Windows 2003 Enterprise Edition R2 or Windows 2008 and 2008R2 Enterprise Edition Active Directory Apache Web Server Apache Web Server (HTTPD) v2.2.4 Apache WebServer Blue Coat ProxySG Syslog Blue Coat ProxySG running SCOGS v5.4, v6.1-6.3.0 WebProxy BMC Remedy Action Request (AR) System BMC Remedy AR System 7.0 on Microsoft Windows 2000 or 2003 Server BMC Remedy ARS CA SiteMinder - Access Management System eTrust SiteMinder 5.5, 6.0 SP1 or SP2 on Windows 2000 with SP4, 2003, or Solaris 8 or 9 Access Control Check Point Firewall (CP Audit) 4.0 SP8, 4.1 SP6, NGAI R55, NGX R65 Firewall Cisco ASA Adaptive Security Appliance v7.2, v8.0, and v8.2 - 8.4 UTM Cisco Content Engine Content Engine with Cisco Application and Content Networking System (ACNS) 4.2 or 5.5 Cisco Content Engine Cisco ESA v7.0, 7.1 Mail Security Cisco Firewall Services Module (FWSM) v4.0, v4.1 and v4.1(7) Firewall/VPN Cisco IOS 12.x, v15.0(M), 15.1(M) Router & Switches Cisco IPS Cisco IPS 4200 running IPS v6.2 or v7.0 IPS Cisco Identity Services Engine (ISE) v1.0.2 Access Control TIBCO LogLogic® Unity User's Guide 74 Log Source Versions/Platforms Device Category Cisco NetFlow Cisco NetFlow v5 or v9, NSEL. IOS XE v15.1(3)M NAT64 NetFlow v9 Router Cisco (Nexus) NX-OS v8.3 Switch Cisco Secure ACS v4.1, v4.2 and v5.2 Access Control Cisco VPN3000 - VPN Cisco Web Security Appliance (WSA) Async OS v6.3 and v7.1 Web Security Fortinet (FortiOS) FortiOS 4.0 MR2, v5.0 Firewall F5 BIGIP Traffic Management Operating System (TMOS) ASM v11.0.0 LTM v11.0.0 Firewall LoadBalancer GuardiumSQLGuard v6.1 DB IDS/IPS GuardiumSQLGuard Audit v6.1 DB IDS/IPS HP NonStop HP NonStop running D48 or later on a K-series System; G06.20 or later on an S-series System; H06 or later on an Integrity NonStop System System HP-UX Operating System Audit HP-UX Audit v11iv2 -11i.31 System Audit IBM AIX Audit v5.3, v6.0, and v6.1. v7.1 System IBM AIX Operating System v5.3, v6.0, and v6.1. v7.1 System IBM DB2 Universal Database (UDB) IBM DB2 UDB v8.1, v8.2, v9.0, v9.5, v9.7 Enterprise Server Edition on Windows, Solaris, HP-UX, Linux, or AIX Database IBM Resource Access Control Facility (RACF) SMF record types 80, 81 and 83. RACF on z/OS 1.6, 1.7, 1.8, 1.9, 1.10, 1.11-1.13 Access Control IBM ISS SiteProtector v2.0 Sp5.0,5.1,6.1, 6.2, 8.0 and 9.0 IPS Juniper IDP v5.0 IDS/IPS Juniper RT_FLOW JunOS v9.3 Firewall TIBCO LogLogic® Unity User's Guide 75 Log Source Versions/Platforms Device Category Juniper SSL VPN Secure Access v5.5, v6.0 R3, v6.1 R1, v6.2, v6.5, v7.0, v7.1 VPN Juniper (JunOS) JunOS v9.3, v10.3 & 10.4 UTM LogLogic Appliance All Platforms System LogLogic Database Security Manager v4.1 Database McAfee ePolicy Orchestrator ePO v4.0, v4.5, v4.6.0, v4.6.1, v4.6.2; HIPS v7.0, v8.0 IPS McAfee G2 Sidewinder FW (v6.1, v6.2, v7.x, v8.0-8.3) Firewall/VPN Microsoft DHCP DHCP Service on Win 2003 or 2003 R2 with SP1 or SP3 DHCP Service on Win 2008 or 2008 R2 with SP1 Microsoft DHCP Application Microsoft Office SharePoint Server Microsoft Office SharePoint 2007, 2010 Content Management Microsoft Operations Manager MOM 2005 SP1 running on Windows 2003 Server MOM 2007 running on Windows 2003/2008 Server System Microsoft Internet Authentication Service (IAS) Microsoft Windows Servers Access Control Microsoft SQL Server Microsoft SQL Server 2005/2008/2012 (Application Logs) Database Microsoft SQL Server SQL Server 2005, 2008, 2012 Standard or Enterprise Database Microsoft Windows Server Windows 2003R2 Server, and Windows Server 2008/R2/2012 Windows Server Microsoft Windows Server (French) Windows 2003 Server and Windows 2008R2 Server Windows Server Microsoft Windows Server (German) Windows 2003 Server and Windows 2008 R2 Server Windows Server Microsoft Windows Server (Japanese) Windows 2003 Server and Windows 2008 R2 Server Windows Server MySQL Server GDBC v5.5.9 Database TIBCO LogLogic® Unity User's Guide 76 Log Source Versions/Platforms Device Category NetApp Decru DataFort DataFort FC-series, E- and Sseries appliances Decru Datafort NetApp Filer NetApp Data ONTAP 7.0,7.3 & 8.0 on FAS900, FAS200, F800, GF900, GF800, NearStore R200, 150, and 100, and F87. (Not supported on F700 or F85.) NetApp Filer, NetApp Filer Audit Novell eDirectory eDirectory 8.8 on Windows 2000 Server with Service Pack 4; Windows 2003 Server Enterprise Edition with Service Pack 1; Windows XP Professional with Service Pack 2; Red Hat Linux Advanced Server 4; or Novell NetWare 6.5 Support Pack 7 LDAP Directory Service Oracle Database Server Oracle 10g R1/R2 or 11g, 10.2.0.4g, 11.2.0.1.0g installed, running on Linux (Fedora Core 3), Solaris 9 (64-bit SPARC and Intel i386), HP-UX 11i, or AIX 5.3 Database Other UNIX AIX 5L, HP-UX 11i v2, Solaris 8/9/10, RHEL 4/5 System General Database Collector for Oracle Oracle 11g, 10.2, 10.1, 9.2 running on Solaris 9/10 Database Palo Alto Networks PanOS Palo Alto Networks PanOS v2.1, v3.0, v3.1.0, v4.0.0-4.0.3, v4.1-4.17.v5.0 UTM RSA ACE/Server ACE/Server 4.x, 5.x, and 6.x on Solaris Access Control Reuters KondorPlus All versions Application Snort v2.4, v2.6, v2.8, v2.9 Intrusion Detection Sourcefire Sensor v4.1 or v4.6, v4.7 - v4.10 IDS/IPS Sourcefire Defense Center v4.9.1.7, 4.10.0.0. v5.0.0-5.1.0, 5.2.0 IDS/IPS Squid2 All versions Blue Coat Sun Solaris Basic Security Module (BSM) Solaris 8/9/10 on Sun SPARC or Intel i386 platforms Sun Solaris Operating System BSM TIBCO LogLogic® Unity User's Guide 77 Log Source Versions/Platforms Device Category Sybase Adaptive Server Enterprise (ASE) Sybase ASE 12.5 or 15.5 on Win XP Pro, Win Svr 2003 Standard or Enterprise; Red Hat Enterprise Linux 4; SUSE Linux Enterprise Server 9; or Sun Solaris 8, 9 or 10 (32 or 64-bit SPARC or Intel i386) platforms, v15.7 Database Symantec SEP v11 and v12 AntiVirus TIBCO ActiveMatrix Administrator V6.3.0 Management Server TIBCO ActiveMatrix BPM v3.0 Business Process TIBCO ActiveMatrix BusinessWorks V5.11 Business Process TIBCO Administrator V5.7.0 Management Server TIBCO API Exchange Gateway Server V2.1 Business Process TIBCO Hawk Agent V5.11 Business Process TIBCO Enterprise Messaging Service Collector (EMSC) v6.3.0 EMS TrendMicro Control Manager v5.0 AntiVirus TrendMicro OfficeScan v10.0 & v10.5 AntiVirus Tripwire for Servers Tripwire for Servers 6.5 with Tripwire Manager 4.6 running on Windows 2003 Enterprise Edition R2 Tripwire Management Station VMware ESX Server VMware ESX v4.0.0, v4.1.0 or v5.0 Hypervisor VMware vCenter VMware vCenter Server v4.0.0, v4.1.0, 5.0.0 and v5.1.0 Managment Server VMware vCenter Orchestrator v4.0.0, 4.1.0, and 4.2.0, v5.1.0 Automation Server VMware vCloud Director VMware vCenter Cloud Director v1.0.1 through v1.5 Management Server VMware vShield Edge VMware vShield Server v4.0.0, v4.1.0 or v5.0.0 Firewall TIBCO LogLogic® Unity User's Guide 78 Filter Syntax The following syntactic rules must be followed while constructing filter expressions. ::= ; ::= ( OR )*; ::= ( AND )*; ::= ( NOT )? ( | | | | | | | | ); ::= “(“ “)”; ::= ; ::= IN “(“ “)” | IN ; ::= BETWEEN AND ; ::= IS ( NOT )? NULL; ::= ( NOT )? LIKE ; ::= ; ::= ( NOT )? REGEXP ; ::= TRUE | FALSE | | ; ::= “=” | “<>” | “!=” | “<” | “<=” | “>” | “>=”; ::= ( ( “+”|”-“|”*”|”/”|”||” ) )* ::= ()? | ()? | | ()? | | ::= “+” | “-“ TIBCO LogLogic® Unity User's Guide