Proximity 4.3 Tutorial

Proximity 4.3 TutorialProximity 4.3 Tutorial Published November 15, 2007 Copyright © 2004-2007 David Jensen for the Knowledge Discovery Laboratory The Proximity Tutorial, including source files and examples, is part of the open-source Proximity system. See the LICENSE file for copyright and license information. All trademarks or registered trademarks are the property of their respective owners. This effort is or has been supported by AFRL, DARPA, NSF, and LLNL/DOE under contract numbers F30602-00-2-0597, F30602-01-2-0566, HR0011-04-1-0013, EIA9983215, and W7405-ENG-48 and by the National Association of Securities Dealers (NASD) through a research grant with the Univeristy of Massachusetts. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation hereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements either expressed or implied, of AFRL, DARPA, NSF, LLNL/DOE, NASD, the University of Massachusetts Amherst, or the U.S. Government. The example database used to support the exercises in this tutorial, ProxWebKB, was developed from the publicly available WebKB relational data set developed by the Text Learning Group at Carnegie-Mellon University. The version used for the Proximity tutorial has been modified from the original distribution to meet the needs of this tutorial. The original dataset is available from www-2.cs.cmu.edu/~WebKB/. General inquiries regarding Proximity should be directed to: Knowledge Discovery Laboratory c/o Professor David Jensen, Director Department of Computer Science University of Massachusetts Amherst, 01003-9264Table of Contents 1. Introduction ............................................................................................................ 1 Conventional Knowledge Discovery ....................................................................... 1 Relational ........................................................................... 1 Proximity Advantages .......................................................................................... 2 2. Getting Started with Proximity ................................................................................... 3 Overview ........................................................................................................... 3 Using the Tutorial ............................................................................................... 3 Proximity .................................................................................................. 4 Contact information ........................................................................................... 10 Tips and Reminders ........................................................................................... 10 3. Importing and Exporting Proximity Data .................................................................... 11 Overview ......................................................................................................... 11 Importing XML Data ......................................................................................... 11 Transforming Tabular Data to XML ..................................................................... 16 Exporting Data to XML ...................................................................................... 17 Importing Plain Text Data ................................................................................... 19 Exporting Plain Text Data ................................................................................... 22 Specialized Data Export ..................................................................................... 23 Deleting Proximity Databases .............................................................................. 24 Tips and Reminders ........................................................................................... 24 4. Exploring Data ...................................................................................................... 27 Overview ......................................................................................................... 27 The Proximity User Interface ............................................................................... 27 Exploring Objects and Links ............................................................................... 28 Exploring Attributes .......................................................................................... 32 Using the Location Bar ...................................................................................... 35 Visualizing Data ............................................................................................... 36 Setting Display Preferences ................................................................................. 40 Analyzing the Database Schema .......................................................................... 41 Tips and Reminders ........................................................................................... 43 5. Querying the Database ............................................................................................ 45 Overview ......................................................................................................... 45 A First Proximity Query ..................................................................................... 46 Exploring Containers and Subgraphs ..................................................................... 50 Grouping Elements in a Query ............................................................................. 56 Comparing Items in a Query ................................................................................ 59 Matching Complex Subgraphs with Subqueries ....................................................... 61 Adding Links to Data with Queries ....................................................................... 64 Executing a Query from the Proximity Database Browser ......................................... 66 a from the Command Line ........................................................... 68 Querying Containers .......................................................................................... 69 Tips and Reminders ........................................................................................... 70 6. Using Scripts ........................................................................................................ 73 Overview ......................................................................................................... 73 Working with Scripts ......................................................................................... 73 Running Proximity Scripts .................................................................................. 74 Using the Python Interpreter ................................................................. 75 Sampling the Database ....................................................................................... 79 Adding a New Attribute ..................................................................................... 81 Social Networking Algorithms ............................................................................. 83 Working with Proximity Tables ........................................................................... 86 Synthetic Data Generation .................................................................................. 94 Tips and Reminders ......................................................................................... 100 iiiProximity 4.3 Tutorial 7. Learning Models ................................................................................................. 101 Overview ....................................................................................................... 101 The Modeling Process in Proximity .................................................................... 101 Relational Bayesian Classifier ........................................................................... 102 Probability Trees .............................................................................. 105 Relational Dependency Networks ....................................................................... 115 Tips and Reminders ......................................................................................... 121 A. Proximity Quick Reference ................................................................................... 123 MonetDB Server ............................................................................................. 123 Proximity Shell Scripts and Batch Files ............................................................... 123 Query Editor Keyboard Shortcuts ....................................................................... 124 Proximity Python Interpreter Commands ............................................................. 125 Location Bar Path Syntax ................................................................................. 125 DTD Files ...................................................................................................... 126 Technical Support and Documentation ................................................................ 126 B. Installation ......................................................................................................... 127 Obtaining Proximity ........................................................................................ 127 Installing MonetDB ......................................................................................... 127 Proximity ......................................................................................... 129 Updating MonetDB Databases ........................................................................... 129 C. Proximity XML Format ......