Quickstart_1.0

Transcript

Eclipse plug-ins for pattern-based instrumentation of applications  Getting Started Guide  Bernhard Jungk Version: December 14, 2007 Contents 1 2 3 4 Introduction 1.1 1.2 1.3 1.4 General architecture Patterns . . . . . . . Log4J . . . . . . . . ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation 3 3 4 4 4 6 2.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Installation using the update site . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Alternative installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Importing the example project 11 Using the plug-ins 4.1 4.2 4.3 4.4 4.5 Add/Congure the instrumentation generation nature Instrumenting with Log4J . . . . . . . . . . . . . . . . Instrumenting with ARM . . . . . . . . . . . . . . . . Code generation . . . . . . . . . . . . . . . . . . . . . Instrumentation Patterns View . . . . . . . . . . . . . 4.5.1 Adding patterns . . . . . . . . . . . . . . . . . 4.5.2 Deleting patterns and points . . . . . . . . . . 4.5.3 Moving and Copying patterns and points . . . 4.5.4 Changing the view . . . . . . . . . . . . . . . . 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 13 14 19 27 29 29 30 30 31 1 Introduction Instrumentation is essential for quality assurance during application development as well as for production system monitoring. Especially for enterprise applications, a detailed analysis of internal processes is important. Depending on the required type of monitoring information, instrumentation can be achieved by inserting general purpose logging statements (e.g. complying to the Log4J [Gul03] specication) or by using specialised performance measurement APIs, e.g. Open Group Application Response Measurement [TOG04] (ARM). The Eclipse plug-ins for model-based instrumentation of applications are a suite of easy to use plug-ins for instrumentation with Log4J and ARM. The tedious task of manual instrumentation is supported by a GUI interface. This guide shows how to instrument a small example application with the help of the instrumentation suite. It starts with the plug-in installation and continues with a description of the instrumentation process for the supplied example application with Log4J and ARM. The nal step is to generate the instrumentation code, which will be presented last. Some basic knowledge of the architecture is important for a successful instrumentation. Thus, the following section discusses the general architecture of the plug-ins, the use of instrumentation patterns and what can be achieved with Log4J and ARM. 1.1 General architecture The general architecture of the instrumentation suite consists of three types of Eclipse features (see g. 1.1). The Core feature acts as the central controller component for the instrumentation suite. The UI uses the core to access the instrumentation model. Backends extend the core to provide support for specic instrumentation APIs (e.g. ARM). Each Backend contains the ability to generate code compatible with an instrumentation API. Furthermore, each Backend provides a number of instrumentation patterns that may be used in conjunction with the backend. Figure 1.1: Eclipse features of the instrumentation suite suite 3 Figure 1.2: Selection of Instrumentation Patterns 1.2 Patterns The Eclipse-based instrumentation suite provides a number of so called basic instrumentation patterns (see g. 1.2). Basic patterns represent basic instrumentation entities that can be combined to form complex instrumentation patterns: A single logging statement is seen as an Event, an ARM measurement with Start an Stop points is forms an Action. Each pattern has one or more roles. Each role denes the purpose of a point or basic pattern in greater detail. All possible roles and participating patterns have to be specied by the Backend. For example, the RPC instrumentation pattern combines a client side Action with a server side Action. The ARM backend represents this complex pattern using two correlated ARM measurements. 1.3 Log4J Log4J is a widely used logging package. The Log4J Backend provides support for Event and Action patterns as shown in g. 1.2. It does however not support the RPC pattern. The generated code for the Event pattern is a simple logging statement: 1 Logger . getLogger ( $logger ) . $priority ( $message ); It may be parametrized by specifying a log message, a priority and a custom logger class. The generated code for an Action pattern looks quite similar. The generated code only diers in the generated log message. For an action, the Log4J Backend adds the action name to the beginning and the role type (Start or Stop ) to the end of the log message. 1.4 ARM ARM is a much more sophisticated API than Log4J. ARM has been designed to measure the performance of marked source code sections (units of work) within an application. The ARM Backend supports Action and RPC patterns. Event patterns are not supported by this 4 backend. The ARM Backend requires an additional Initalization pattern, which is used to mark the source code positions for ARM implementation initalization. The backend generates a helper class (the EmivArm class), which eases the generation of code inlined at the actual instrumentation points. The actual generated code for Action Start and Stop roles is 1 // start role $exportCorrelator = EmivaArm . getInstance () . startTransaction (" $id " , " $transactionName " , new EmivaArm . ContextProperties ( $contextProperties ) , $importCorrelator ) ; 3 // stop role 5 EmivaArm . getInstance () . stopTransaction (" $id "); The code for the Stop role is simpler. It does not require user specied attributes, because the id is generated by the ARM Backend. The code for the Start role is much more interesting. It may be specialized with a transaction name and several user dened context properties. Furthermore, the user can specify import and export correlators, which are used by the correlation feature of ARM. The user still has the responsibility to manage the correlators in a meaningful way, e.g. transfer the correlator to the server side Action of an RPC pattern. The RPC pattern does not have an eect on the code generation. 5 2 Installation There are two ways to install the plug-ins. The preferred way is to use the Eclipse update site. 2.1 Prerequisites The instrumentation suite has the following prerequisites: • Java SE v5 • Eclipse v3.3 with JDT • An ARM implementation (e.g. OpenGroup ARM1 ) to use the ARM Backend The instrumentation suite may run on dierent congurations as well, but they are unsupported. 2.2 Installation using the update site Select Help -> Software Updates -> Find and Install in the menubar (see g. 2.1). Figure 2.1: Find and install menu 1 http://www.opengroup.org/arm/ 6 In the following dialog, choose Search for new features to install and click Next (see g. 2.2). Figure 2.2: Search for new features to install dialog Click on New Remote Site (see g. 2.3). Figure 2.3: Update sites overview 7 Another dialog opens (see g. 2.4). Choose a name for the update site (e.g. Instrumentation Generation), enter the URL http://wwwvs.informatik.fh-wiesbaden.de/downloads/extern/oss/permanenta/update-site/ and click Ok. Figure 2.4: New update site Now select the entry with the name you have chosen and click on Finish. After a short while a new dialog opens (see g. 2.5). Figure 2.5: Selection of features to install You have to choose which features you want to install. Required are at least the Core, the UI and one of the ARM or Log4J features. To complete all parts of the guide, you have to install all available features. Click on Next, once you have selected the features to install. Accept the license and click Next. 8 The last dialog page lists the features you have selected to install (see g. 2.6). Figure 2.6: Conrmation of features to install Once Finish has been pressed, Eclipse will start to download the necessary les from the update site. When the download has nished, Eclipse will ask, if you really want to install the selected features, because the digital signature can not be veried (see g. 2.7). Figure 2.7: Verication of the digitally signature failed. Click on Install All. Usually Eclipse will ask you, if you want to restart Eclipse. If you do not need to save your work, you should choose Yes. When Eclipse has been restarted, you may proceed with importing a small example project into your workspace. 9 2.3 Alternative installation The alternative way is to download a tar-gz package (instrumentation-tools.tgz2 ) and extract the content to the plugin directory of the local eclipse installation. The default location of the plugin directory is eclipse/plugins, where eclipse is the directory of your local Eclipse installation. 2 http://wwwvs.informatik.fh-wiesbaden.de/downloads/extern/oss/permanenta/ instrumentation-tools.tgz 10 3 Importing the example project The guide uses the example project to show how to use the instrumentation suite. The example project is a small application with one or more client threads and a server thread. The clients send messages to the server, which receives and prints the messages to standard out. Both parties wait a small random period to simulate workload during each pass. Download the project from the download location1 , extract and import it into your Eclipse workspace. To do that, open the context menu in the package explorer and click on import (see g. 3.1). Figure 3.1: Project import with the Package Explorer In the import dialog, select General -> Existing Projects into Workspace and click Next (see g. 3.2). Figure 3.2: Import dialog 1 http://wwwvs.informatik.fh-wiesbaden.de/downloads/extern/oss/permanenta/example-project.tgz 11 Press the Browse-Button on the right side of Select root directory (see g. 3.3). Figure 3.3: Import dialog Browse to the directory where you have extracted the project. Once you have selected the directory, the getting_started project should show up in the import dialog. You may choose to copy the project into your workspace, but this is not mandatory. Click on Finish. Now, you have successfully imported the example project. 12 4 Using the plug-ins After the successful installation of the plug-ins and the import of the example project, you can to start using the instrumentation suite. This guide will show you how to add the instrumentation generation nature to your project, and how to instrument the example project with Log4J and ARM. This guide also shows how to generate instrumentation code for the instrumentation points you have created in the instrumentation steps. 4.1 Add/Congure the instrumentation generation nature First, you need to add the instrumentation generation nature to the example project. Select the example project in the package explorer. Open the context menu and click on Instrumentation Generation -> Add Instrumentation Generation (see g. 4.1). Figure 4.1: Adding the instrumentation generation nature to a project. Now you are asked where to store the repository. One repository is the central instrumentation model storage. For simplicity, choose the imported example project (see g. 4.2). Figure 4.2: Dialog for choosing the repository location. The repository is used for all projects in the workspace. You can only have one repository per workspace. 13 Workspace-wide settings like the repository location can be changed in the project settings dialog of a project with instrumentation generation nature. Open the properties dialog for the example project. You can do that with the context menu in the package explorer. In the dialog, select the Instrumentation Generation tab (see g. 4.3). Figure 4.3: Instrumentation Generation properties. 4.2 Instrumenting with Log4J In order to use Log4J, the project needs to access a Log4J jar le in the classpath. The instrumentation suite contains a suitable Log4J jar, which can be automatically copied to all projects instrumented with Log4J. Figure 4.4: Properties: copying the Log4J library to instrumented projects. However, the suite does not copy the jar by default. You have to activate the copy option in the project properties dialog (described in the previous section). In this dialog, you have to activate the Add Log4J library to project checkbox (see g. 4.4). Now you can start to instrument the example application. The most primitive pattern one can apply is an Event pattern. This will be the rst pattern you create. Open the World.java le in the example project. Change the cursor position to the rst statement in the main method: The Log4J Event pattern 14 1 int numClients = 2; Now right click to open the editor's context menu. Select Instrumentation Generation -> Add Instrumentation Point (see g. 4.5). Figure 4.5: Adding an instrumentation point. The Create new point dialog opened (see g. 4.6). Figure 4.6: The Create new point dialog. In this dialog, you can create new instrumentation points and new basic patterns. 15 Create a new basic pattern, by pressing the New basic pattern button. Choose a name, e.g. Application startup (see g. 4.7). Figure 4.7: Choosing a pattern name. Then you should select Event (Log4JBackend) and click OK (see g. 4.8). Figure 4.8: Choosing a pattern type. Back in the Create new point dialog, you can see the newly created pattern. To nally create an instrumentation point, you have to select the empty Source role in the pattern and click OK (see g. 4.9). Figure 4.9: A new pattern. 16 You should see a blue L4 with a red cross (see g. 4.10). Figure 4.10: A new instrumentation point. The red cross indicates that the pattern is deactivated. Each pattern is initially deactivated to prevent the generation of incomplete code, in case required properties or related instrumentation entities are missing. The next step is to add a log message to the instrumentation point. Change the cursor position back to the instrumentated statement. Open the context menu and select Instrumentation Generation -> Properties (see g. 4.11). Figure 4.11: Opening the properties view for a point. Eclipse will open the properties view. You can enter a log message in the log4j.message property, e.g. Application startup successful. In the properties view, you may change the Log4J logger and the priority, too (see g. 4.12). Figure 4.12: Changing the Log4J log message. Now continue with additional patterns. You can add two Action patterns, one for the lifetime of the client threads and one for the lifetime of the server thread. The Log4J Action pattern Select the statement, where the client threads are started: 1 clients [ i ]. start () ; Open the Create new point dialog again and add a Log4J action pattern. You can name it Client lifetime for example. The Action pattern for Log4J is Action (Log4JBackend). You should now see your newly created action pattern in the Create new point dialog. Select the Start role and click OK. 17 The newly created instrumentation point should show up similar to the rst instrumentation point. The next step is to create an instrumentation point for the Stop role. Change the cursor position in the editor to the join statement for clients: 1 clients [ i ]. join () ; Open up the Create new point dialog, select the Stop role and click OK (see g. 4.13). Figure 4.13: Adding the stop point. The last step is to change the log messages for both points of the Action pattern. This works exactly as described for the event pattern. The main dierence is that the name of the Action pattern is added to the beginning of the message, and the name of the role is added at the end. Therefore, you can choose for example Client as log message. This will be changed to Client lifetime: Client (Start) for the start role and Client lifetime: Client (Stop) for the stop role. This message is not very meaningful, yet. You may wish to add the Client thread id to the log message. This is achieved by changing the log message to Client $clients[i].getId()$. Everything between the $-signs will be treated as code, which is concatenated to the message string at the position in the log message property. You can now add another Action pattern for the server thread or go on with the next section. 18 4.3 Instrumenting with ARM Instrumenting with ARM is similar to the Log4J case. In order use ARM, the project needs ARM implementation libraries and the Open Group ARM interfaces in the classpath. The ARM interfaces are supplied with the instrumentation suite combined with the OpenGroup null implementation. The null implementation makes sure you can run an ARM instrumented application, but does not do anything further. There is a more functional implementation available at http://www.opengroup.org/arm/. If you have a dieent ARM implementation available, you can use this one. After you have decided which ARM implementation to use, you may open the project properties as described in section 4.1. Figure 4.14: Properties: adding the ARM null implementation to the project. There you have to enter the name of the transaction factory corresponding to your ARM implementation. If you have chosen to use the null implementation, you may additionally check the Add OpenGroup null implementation checkbox, as this implementation is packaged with the instrumentation suite and can automatically be copied to the application classpath (see g. 4.14). ARM needs some conguration work to be done, before it can be used. This is the task of the Initalization pattern. Open the World.java le. There is no suitable statement to add the Initalization pattern to, so we will create one. Insert a single ; somewhere before the following statement: The ARM Initalization pattern 1 server . start () ; This is necessary, because you cannot add an instrumentation point to an empty line. The ; is an empty statement, and therefore you can use it, to add an instrumentation point to an empty line. 19 Set the cursor at the ; statement. Open up the Create new point dialog as described in 4.2 (see g. 4.15). Figure 4.15: The create new point dialog. Create a new basic pattern, by pressing the New Basic Pattern button. Name the initalization pattern something like ARM initialization (see g. 4.16). Figure 4.16: Enter a name for the new pattern. Next, you have to choose the Initalization (ArmBackend) pattern (see g. 4.17). Figure 4.17: Choose the initialization pattern. 20 Select the role of the initalization pattern and click OK in the Create new point dialog (see g. 4.18). Figure 4.18: The create new point dialog. You should now see a yellow arrow on the left of the ;, which has a red cross painted over it. The red cross indicates, that the instrumenation point is deactivated (see g. 4.19). Figure 4.19: The deactivated initialization point. After you have added the Initalization point, you have to edit the properties of the point. Open up the properties view for the instrumentation point, using the context menu of the editor (Instrumentation Generation -> Properties) (see g. 4.20). Figure 4.20: The editor context menu to open the properties view. 21 In the properties view you have to enter an application name, e.g. 4.21). Hello World! (see g. Figure 4.21: The properties view. If you did not change the ARM transaction factory property to use the transaction factory of your ARM implementation, you can do that now. The initialization point is now complete. You will now learn about another way to work with patterns. Note that to create basic patterns, it's possible to use two ways. In the Log4J chapter and for the Initalization you learned how to create a basic pattern with the Create new point dialog. Now, you will learn how to create patterns with the Instrumentation Patterns view. The ARM Action pattern Open the Instrumentation Patterns view, by choosing Show View -> Others in the Window menu (see g. 4.22). Figure 4.22: Window menu and Show View sub menu. In the Show View dialog, choose Instrumentation Generation -> Instrumentation Patterns and click on OK. Figure 4.23: The Show View dialog. 22 The Instrumentation Patterns view is now visible somewhere in your workbench. In that view you see the patterns, you have created already (see g. 4.24). Figure 4.24: The Instrumentation patterns view. Before you create the rst pattern with this view, consider that the aim is to create an RPC pattern. An RPC pattern is created by combining two Action patterns. Therefore you should start by creating two Action patterns. One pattern on the client side and another one on the server side of the application. To create a new Action pattern in the view, click on the little plus sign in the Instrumentation Patterns view's toolbar (see g. 4.25). Figure 4.25: The green plus sign to add patterns. Choose a name for the client side 4.26). Action pattern, e.g. Client action Figure 4.26: Choose a name for the pattern. 23 and click OK (see g. Then you have to select Action (ArmBackend) and click OK again (see g. 4.27). Figure 4.27: Choose a ARM Action pattern. You will see the new pattern in the Instrumentation Patterns view. It will have two missing roles (see g. 4.28). Figure 4.28: Instrumentation Patterns view with the new pattern. Create another pattern following the same instructions for the server the server pattern Server action. Action pattern. Name Open the Client.java le now. Set the cursor position in the editor to the following statement: 1 message = new HashMap < String , byte [] >() ; Then open the editor context menu and choose Instrumentation Generation -> Add Instrumentation Point. In the Create new point dialog, select the Start role of the Client action pattern and click OK. There is no suitable statement for the Stop role but every instrumentation point is bound to a statement. You can create a empty statement with ;. Add an empty statement behind the rst catch block and create an instrumentation point at this empty statement but select the Stop role of the Client action in the Create new point dialog. Continue with the Server.java le. Use this statement as the Start role for the Server action : 1 System . out . println ( message . keySet () . iterator () . next () ); and create an empty statement after the catch block. Use this empty statement for the Stop role. 24 The next step is to change the properties of each Start role to suitable values. Begin with the Client action. Open the Instrumentation Patterns view again. Scroll to the Start role of the Client action and double click on the role. The Properties View opens. The client side does not have any import correlator. Instead it needs some export correlator, which has to be transfered to the server side. The server uses this correlator as import correlator. Enter a name for the correlator, e.g. exportCorrelator (see g. 4.29). Figure 4.29: The ARM client Action properties. The generated code uses a variable with this name to store the correlator. Therefore you have to add the declaration of the exportCorrelator before the instrumented statement: 1 byte [] exportCorrelator = null ; 3 message = new HashMap < String , byte [] >() ; The example is prepared to transfer the correlator with the message to the server. To do that, change this statement: 1 message . put (" pid : " + mPid + " , iteration : " + i , new byte [0]) ; to 1 message . put (" pid : " + mPid + " , iteration : " + i , exportCorrelator ); The client side instrumentation is nished now. Continue with the properties of the Start role of the Server action. Open the Properties View for the Start role of the Server action. Change the import correlator to some name like importCorrelator (see g. 4.30). Figure 4.30: The ARM client Action properties. The import correlator has to be extracted from the received message. To do so, insert the following statement: 25 1 while (! message . containsKey (" shutdown ")) { 3 + byte [] importCorrelator = message . entrySet () . iterator () . next () . getValue () ; 5 // process the message System . out . println ( message . keySet () . iterator () . next () ); The server side instrumentation is nished now, too. The next step is to combine both Action patterns to an RPC pattern. Open the Instrumentation Patterns view. Create a new pattern by clicking on the green plus symbol in the view's toolbar. Enter a meaningful name, e.g. RPC. Choose the RPC (ArmBackend) pattern. The new RPC pattern should show up in the Instrumentation Patterns view (see g. 4.31). The ARM RPC pattern Figure 4.31: The new RPC pattern. Unfold the RPC pattern and use drag and drop to move the Client and Server Action pattern into the RPC pattern. The Client pattern should be dropped at the ClientSide role, the Server pattern at the ServerSide role (see g. 4.32). Figure 4.32: The complete RPC pattern. You nished the creation of a RPC pattern. 26 4.4 Code generation Code can only be generated for activated instrumentation points. The default is to leave patterns deactivated after creation, until they are explicitly activate. Therefore, you have to activate all the patterns and instrumentation points you created earlier, before you can generate code for them. There are several ways to activate the instrumentation points: Pattern activation • At project, package and le level • At pattern level • At point level For activating all instrumentation points of a project, of a package or of a single le, you have to select the corresponding entity in the package explorer, open the context menu and choose Instrumentation Generation -> Activate instrumentation points (see g. 4.33). Figure 4.33: Activation of instrumentation points with the package explorer. For activating all instrumentation points belonging to a pattern, you have to open the Instrumentation Patterns view, select the desired pattern and use the context menu to activate that pattern (Activate/Deactivate, see g. 4.34). Figure 4.34: Activation of instrumentation points with the instrumentation patterns view. 27 For activating a single point, you can either use the Instrumentation Patterns view in the same way as for patterns, but instead of a pattern you have to select an instrumentation point. Another way is to use the editor context menu (Instrumentation Generation -> Activate instrumentation point, see g. 4.35). Figure 4.35: Activation of single instrumentation points with the editor context menu. If a pattern is not complete, the user will be warned with a dialog message (see g. 4.36). Figure 4.36: Warning for activation of incomplete patterns. The deactivation works very similar in all three cases, you just have to select Deactivate instrumentation points instead of Activate instrumentation points. The code generation can be triggered in the same granularity as the activation and deactivation of patterns: at project, pattern or point level. You have to choose Generate Instrumentation Code instead of Activate Pattern/Point in the context menus that also oer the activation selection. The removal of generated code works accordingly, replacing Generate Instrumentation Code with Remove Instrumentation Code. Code generation and removal There are some things to note about the code generation and removal. Not all of the generated code will be removed. You have to do some manual work to remove all generated code for imports, the EmivaArm class and the libraries (ARM, Log4J) copied to the project. If you deactivate a pattern, code that was generated for the patterns before gets removed. The code is rebuilt, if you activate the pattern again. 28 4.5 Instrumentation Patterns View The Instrumentation Patterns view is one of the central places to do the instrumentation work. Therefore, it is important to know the details of this view. 4.5.1 Adding patterns Adding patterns is done in four simple steps: • Click on the green plus symbol (see g. 4.37) Figure 4.37: The green plus sign, to add patterns. • Choose a name (see g. 4.38) Figure 4.38: Choose a name for the pattern. • Choose a pattern (see g. 4.39) Figure 4.39: Choose the desired pattern. • Add patterns and points for the roles as required The fourth step may require additional work in the editor context menu, because you can only create new instrumentation points in the editor. 29 4.5.2 Deleting patterns and points Deleting patterns is achieved using the context menu. Just select the pattern or point you wish to delete and choose Delete in the context menu. If you delete the last instrumentation point of a pattern, the whole pattern is deleted as well (see g. 4.40). Figure 4.40: Delete a pattern. 4.5.3 Moving and Copying patterns and points Patterns and points can be moved and copied betwenn patterns. This is done by drag and drop of patterns or points. They can only be moved or copied if they satisfy the requirements of the new role (see g. 4.41). Figure 4.41: Moving a pattern with drag and drop. If you want to perform a copy, you have to hold down the Ctrl key while dropping the pattern or point. 30 4.5.4 Changing the view The view can show the patterns in two dierent organizations: • Show all patterns or show patterns per project (see g. 4.42 and 4.43) Figure 4.42: The pattern view, showing all patterns. Figure 4.43: The pattern view, showing patterns per project. Additionally the view may be ltered (see g. 4.44): • Hide empty patterns • Show only complex patterns • Hide basic patterns which are part of a complex pattern. Figure 4.44: The pattern view, ltering options. 31 Bibliography [Gul03] Ceki Gulcu. The Complete Log4j Manual: The Reliable, Fast and Flexible Logging Framework for Java . QOS.ch, Mai 2003. [TOG04] The Open Group. Application Response Measurement Binding, December 2004. Catalog number C042. 32 (ARM) Issue 4.0 V2 - Java