DOCTORAL DEFENSE

July 23, 1999 2:00PM at 3-216 CST

ARCHITECTURE OF FINE-GRAINED DATA FLOW NETWORK PROGRAMMING ENVIRONMENTS

BY

Yuh-Jye Chang

ABSTRACT OF DISSERTATION

This thesis focuses on the architecture of a data flow visual programming environment. An experimental system NeatTools was built to test and verify key concepts. Although the system and architecture considerations are general, software support for assisting interfaces is the main motivation for and test bed for our ideas.

The programming environment had to be implemented based on a programming language before the computer science experiments could proceed. In this thesis, I discuss the pros and cons among C, C++ and Java. Later on I discuss and compare NeatTools with Object Management Environments, Modeling Language, and some commercial visual programming tools.

 

ARCHITECTURE OF FINE-GRAINED DATA FLOW NETWORK PROGRAMMING ENVIRONMENTS

 

 

By

YUH-JYE CHANG

 

 

 

DISSERTATION

 

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Information Science in the Graduate School of Syracuse University

July 1999

 

 

Approved _________________________

Professor Geoffrey Fox

 

Date _____________________________

 

 

 

Copyright 1999 Yuh-Jye Chang

All rights Reserved

CONTENTS

 

1.   Introduction.................................................................................... 1

2.   Detailed Design Considerations of NeatTools................................... 3

2.1     Project Requirements......................................................................................... 3

2.2     C++ vs. Java...................................................................................................... 3

2.2.1     Java Background......................................................................................... 4

2.2.2     Old Benchmark........................................................................................... 5

2.2.3     New Benchmark......................................................................................... 7

2.2.4     Benchmark Analysis.................................................................................. 10

2.2.5     Conclusion................................................................................................ 11

2.3     How to get JAVA without using Java................................................................ 11

2.3.1     C++ Features that Java avoids................................................................... 12

2.3.1.1       Multiple Inheritance................................................................................................................... 12

2.3.1.2       Operator Overloading................................................................................................................. 14

2.3.1.3       Pointer........................................................................................................................................ 15

2.3.1.4       Memory Allocation.................................................................................................................... 16

2.3.1.5       Fragile Super Class..................................................................................................................... 18

2.3.2     Some Features that Java Adds................................................................... 19

2.3.2.1       Build-in Thread and Synchronization Primitives....................................................................... 19

2.3.2.2       Build-in Array............................................................................................................................ 20

2.3.2.3       Cross platform and Windowing API.......................................................................................... 21

2.3.3     Conclusion................................................................................................ 22

2.4     NeatTools Architecture Analysis....................................................................... 23

2.4.1     Module Abstraction................................................................................... 23

2.4.1.1       Data............................................................................................................................................ 25

2.4.1.2       Actions....................................................................................................................................... 27

2.4.1.3       Connections................................................................................................................................ 28

2.4.2     Implementation Concept............................................................................ 29

2.4.3     Benchmark Information on NeatTools........................................................ 32

2.4.4     Benchmark Analysis of NeatTools............................................................. 33

2.4.5     Conclusion................................................................................................ 34

2.5     NeatTools and Object Management Environments............................................ 34

2.5.1     CORBA.................................................................................................... 34

2.5.1.1       Object Definition........................................................................................................................ 35

2.5.1.2       CORBA Provided Object Interoperability................................................................................. 36

2.5.1.3       OMA Provided Application-Level Integration.......................................................................... 36

2.5.2     COM/DCOM........................................................................................... 37

2.5.2.1       Objects and Interfaces................................................................................................................ 38

2.5.2.2       Interface Description Language (IDL)........................................................................................ 38

2.5.2.3       Service Control Manager (SCM)................................................................................................ 38

2.5.3     A Comparison of CORBA and COM/DCOM........................................... 39

2.5.4     Benchmark Information on CORBA.......................................................... 40

2.5.5     Benchmark Analysis of CORBA................................................................ 41

2.5.6     Conclusion................................................................................................ 42

2.6     NeatTools and Modeling Languages................................................................. 43

2.6.1     Colored Petri-Net..................................................................................... 43

2.6.1.1       Introduction................................................................................................................................ 43

2.6.1.2       Why use CP-nets?...................................................................................................................... 43

2.6.1.3       Analysis of CP-nets................................................................................................................... 44

2.6.1.4       Design/CPN................................................................................................................................ 45

2.6.1.5       Conclusion.................................................................................................................................. 46

2.6.2     UML (Unified Modeling Language)............................................................ 47

2.6.2.1       Introduction................................................................................................................................ 47

2.6.2.2       UML definition.......................................................................................................................... 47

2.6.2.3       Development Project Artifacts................................................................................................... 48

2.6.2.4       Programming Languages.............................................................................................................. 49

2.6.2.5       Conclusion.................................................................................................................................. 49

2.7     NeatTools and Other Visual Programming Tools............................................... 49

2.7.1     Microsoft Visual Series on C++, Basic, J++............................................... 49

2.7.1.1       Introduction................................................................................................................................ 50

2.7.1.2       Conclusion.................................................................................................................................. 50

2.7.2     JavaBeans................................................................................................. 51

2.7.2.1       Introduction................................................................................................................................ 51

2.7.2.2       Linkage Model in JavaBeans...................................................................................................... 52

2.7.2.3       Conclusion.................................................................................................................................. 53

2.7.3     LabView................................................................................................... 54

2.7.3.1       Introduction................................................................................................................................ 55

2.7.3.2       Data flow Programming.............................................................................................................. 55

2.7.3.3       Graphical Compiler.................................................................................................................... 55

2.7.3.4       Multithreading............................................................................................................................ 56

2.7.3.5       Conclusion.................................................................................................................................. 56

2.7.4     AVS/Express............................................................................................ 57

2.7.4.1       Introduction................................................................................................................................ 58

2.7.4.2       The Graphics Display Kit.......................................................................................................... 58

2.7.4.3       The Data Visualization Kit......................................................................................................... 58

2.7.4.4       The Database Kit........................................................................................................................ 59

2.7.4.5       Visual Programming.................................................................................................................... 59

2.7.4.6       Conclusion.................................................................................................................................. 59

3.   Summary of NeatTools Implementation.......................................... 60

3.1     Design Goal of NeatTools................................................................................ 60

3.1.1     Simple....................................................................................................... 61

3.1.1.1       Simple Usage for the Data-flow Network Designer................................................................... 61

3.1.1.2       Simple Programming Paradigm for the Module Designer........................................................... 62

3.1.2     Object-oriented......................................................................................... 62

3.1.3     Network-ready......................................................................................... 63

3.1.4     Robust...................................................................................................... 63

3.1.5     Secure....................................................................................................... 64

3.1.6     Architecture Neutral.................................................................................. 64

3.1.7     Portable.................................................................................................... 65

3.1.8     High-performance...................................................................................... 66

3.1.9     Multi-threaded........................................................................................... 66

3.1.10   Dynamic.................................................................................................... 67

3.2     NeatTools' Visual Programming Features.......................................................... 68

3.2.1     Introduction............................................................................................... 68

3.2.2     Multi-Thread Features............................................................................... 69

3.2.3     Keyboard and Mouse Event Simulator/Filter.............................................. 70

3.2.4     Networking and TCP/IP............................................................................ 72

3.2.5     Container Nest Structure (Complex Module)............................................. 73

3.2.6     Transfer Focus among Text Fields.............................................................. 74

3.2.7     Polymorphic Data Type............................................................................. 75

3.2.8     Multimedia Features.................................................................................. 77

3.2.9     Multimedia Database................................................................................. 78

3.2.10   External Module and Dynamic Link Library................................................ 80

3.2.11   State Machine........................................................................................... 81

3.3     NeatTools Architecture.................................................................................... 82

3.3.1     Three-Layer Architecture........................................................................... 82

3.3.2     Package Structure..................................................................................... 85

3.3.3     OS and C++ Runtime Package.................................................................. 86

3.3.4     Java-like API package............................................................................... 86

4.   Experiments.................................................................................. 88

4.1     Case Descriptions............................................................................................ 88

4.1.1     Eyal Sherman............................................................................................ 88

4.1.2     Brooke Kendrick...................................................................................... 89

4.2     Approach......................................................................................................... 89

4.3     Limitations of Early Software............................................................................ 90

4.4     Ideas and improvements................................................................................... 91

4.4.1     The mouse driver prototype....................................................................... 91

4.4.2     Next Generation Neat Software - NeatTools............................................. 92

4.5     Key Issues in Building Better Neat Software..................................................... 93

4.5.1     Java Like Cross Platform API in C++........................................................ 93

4.5.2     Module Coordinate System....................................................................... 94

4.5.3     Module Event Broadcast Model................................................................ 94

4.5.4     Increase Usability...................................................................................... 96

4.5.5     Identify the Modules.................................................................................. 97

4.5.6     Dynamic Ports........................................................................................... 98

4.5.7     Display Feedback Mechanism................................................................... 99

4.5.8     Cyclic Data Flow Network Issues............................................................ 100

4.5.9     Thread and Concurrent Related Issues..................................................... 102

4.6     Experiments................................................................................................... 104

4.7     Results........................................................................................................... 109

4.8     Conclusion..................................................................................................... 110

5.   Conclusion.................................................................................. 111

5.1     Future Development....................................................................................... 111

5.1.1     Aggregate Data Type............................................................................... 111

5.1.2     Connection Visibility................................................................................ 112

5.1.3     Undo/Redo Features............................................................................... 112

5.1.4     Network Support Modules...................................................................... 113

5.1.5     Authentication Modules........................................................................... 113

5.1.6     Visualization Modules.............................................................................. 113

5.1.7     Artistic Modules...................................................................................... 113

5.2     Conclusion..................................................................................................... 114

6.   Appendix.................................................................................... 116

6.1     NeatTools Reference Manual......................................................................... 116

6.1.1     NeatTools Module Specification:............................................................. 116

6.1.2     NeatTools Class Hierarchy:..................................................................... 146

6.1.3     Java-like API package............................................................................. 155

6.1.3.1       LANG package......................................................................................................................... 155

6.1.3.2       UTIL package........................................................................................................................... 156

6.1.3.3       IO package................................................................................................................................ 156

6.1.3.4       NET package............................................................................................................................ 157

6.1.3.5       AWT package........................................................................................................................... 158

6.1.4     NeatTools application package................................................................ 160

6.1.4.1       NEAT package......................................................................................................................... 161

6.1.4.2       Modules package...................................................................................................................... 164

6.1.4.3       External modules package......................................................................................................... 164

6.1.4.4       DESKTOP package.................................................................................................................. 165

6.2     Module Programming Introduction.................................................................. 165

6.2.1     Module Event Broadcast Model.............................................................. 165

6.2.2     Basic Methods in Module........................................................................ 169

6.2.3     Information Methods in Module............................................................... 170

6.2.3.1       JString inputTag(int n), JString outputTag(int n).................................................................... 171

6.2.3.2       int inputType(int n), int outputType(int n)............................................................................ 171

6.2.3.3       boolean inputAllowed(int n), boolean outputAllowed(int n).................................................. 172

6.2.3.4       int inputFace(int n), int outputFace(int n)............................................................................... 172

6.2.3.5       JFRect inputArea(int n), JFRect outputArea(int n)................................................................ 173

6.2.3.6       JFPoint inputPoint(int n, class JLinkObj& link), JFPoint outputPoint(int n, class JLinkObj& link)   174

6.2.4     Display Related Methods in Module......................................................... 175

6.2.5     Module persistency related Methods........................................................ 178

6.2.6     Property Related Methods in Module....................................................... 180

6.2.7     About Polymorphic Data Type................................................................. 182

6.2.8     Use Thread in Module............................................................................. 182

6.2.9     Concurrency Issues when Design Module................................................ 184

7.   Bibliography................................................................................ 186


 

FIGURES

Figure 1: Module Abstraction......................................................................................... 23

Figure 2: Connection...................................................................................................... 28

Figure 3: Implementation Concept (1)............................................................................ 30

Figure 4: Implementation Concept (2)............................................................................ 31

Figure 5: Speed Test...................................................................................................... 33

Figure 6: CORBA......................................................................................................... 35

Figure 7: COM/DCOM................................................................................................. 37

Figure 8: NeatTools Introduction.................................................................................... 69

Figure 9: Multi-thread features....................................................................................... 70

Figure 10: Keyboard/mouse event simulator/filter (1)...................................................... 71

Figure 11: Keyboard/mouse event simulator/filter (2)...................................................... 72

Figure 12: Networking and TCP/IP................................................................................ 73

Figure 13: Complex module........................................................................................... 74

Figure 14: Transfer focus among text fields..................................................................... 75

Figure 15: Polymorphic data type................................................................................... 77

Figure 16: Multimedia features....................................................................................... 78

Figure 17: Multimedia database...................................................................................... 79

Figure 18: External module............................................................................................. 80

Figure 19: State machine................................................................................................ 82

Figure 20: Top-level Packages....................................................................................... 85

Figure 21: OS and C++ Runtime Layer.......................................................................... 86

Figure 22: Java like Cross-Platform API Layer............................................................... 87

Figure 23: Exception Class Diagram in Java Like API Layer........................................... 88

Figure 24: Cyclic Data Flow Network Issues............................................................... 101

Figure 25: Headsets to pickup gesture signal................................................................. 104

Figure 26: Mouse driver prototype (1)......................................................................... 105

Figure 27: Mouse driver prototype (2)......................................................................... 106

Figure 28: Typing utility prototype................................................................................ 107

Figure 29: Joy-Mouse.................................................................................................. 108

Figure 30: FITALY keyboard...................................................................................... 109

Figure 31: Experiment results....................................................................................... 110

Figure 32: NeatTools application layer......................................................................... 160

Figure 33: Property in NEAT package......................................................................... 161

Figure 34: NEAT package (continue)........................................................................... 162

 

TABLES

Table 1: Old benchmark (1)............................................................................................. 7

Table 2: Old benchmark (2)............................................................................................. 7

Table 3: New benchmark (1)........................................................................................... 9

Table 4: New benchmark (2)........................................................................................... 9

Table 5: CORBA benchmark infomation........................................................................ 41

Table 6: Summary of layers............................................................................................ 83

Table 7: Classes of LANG package............................................................................. 156

Table 8: Classes of UTIL package............................................................................... 156

Table 9: Classes of IO package................................................................................... 157

Table 10: Classes of NET package.............................................................................. 157

Table 11: Classes of AWT package............................................................................. 160

Table 12: Classes of NEAT package........................................................................... 164

 

 

1.     Introduction

A workable human computer interface can be a vital asset for disabled users who may require computers for tasks such as data collection, gesture recognition, control of external devices, virtual world control, remote collaboration, and perceptual modulation. To accomplish this, a highly configurable, easy to use, extensible, and functional system is needed. In this thesis, the architecture of a fine-grained data flow visual-programming environment is proposed for experimentation and providing solutions to this challenge.

A lot of efforts have been emphasized on using visual programming to assist the traditional textual programming. For example, the commercial visual series products like Visual C++, J++, and Visual Basic from Microsoft Corp. [8] Others focus on creating the textual programming counterpart of visual programming such as G programming language that was used to develop LabView [23] -- a graphical programming development environment. Or, focus on data visualization by using data-flow networks like AVS/Express [22]. The visual programming architecture in NeatTools is different from all the above. It is not intended to be a visual environment that helps generate the desired application template with textual code. It does not follow the traditional textual programming paradigm such as imitating the for-loop, while-loop, process structure, function call, etc. Nor does it emphasize a particular field of application and try to integrate the coarse-grained modules through data-flow networks. NeatTools architecture is built strictly upon module abstraction which has its roots in the formal Input / Output automaton model of Lynch and Tuttle [1]. Module is the basic building block, which the modular communication is carried out through the logical connections. Base on NeatTools architecture, a generic fine-grained visual-programming environment is implemented from a pure visual programming and high performance perspective. All modules are logically equal because they all derive from the same module base class.

In the next section, I describe the detailed design considerations of NeatTools including project requirements, platform and language issues, architecture analysis, object management environments, modeling languages, and other visual programming tools. I discuss C++ vs. Java, C++ features that Java avoids, and features that Java adds. Also, I analyze NeatTools architecture and implementation concept through the module abstraction.

In section 3, I summarize the design goal of NeatTools to be simple, object-oriented, network-ready, robust, secure, architecture neutral, portable, high-performance, multi-threaded, and dynamic. I also introduce visual programming features such as multi-thread, keyboard/mouse event simulator/filter, networking, container nest structure, polymorphic data type, multimedia, database, external module and state machine. Then, I discuss the NeatTools architecture including the three-layer architecture, package structure, OS and C++ runtime package, and Java-like API package, etc.

In section 4, the experiments were carried out through Eyal Sherman [39] and Brooke Kendrick's [40] case. I have described my approach to these cases, the limitations of early software, my ideas and improvements over the early software, and key issues in building better Neat software. Later, the experiments are presented and encouraging results were achieved.

2.     Detailed Design Considerations of NeatTools

2.1     Project Requirements

(1)   General Human Interface Tool.

(A)  Can adapt to any external industry or in-house devices.

(B)  Able to handle complex, real-time, and calculation intensive tasks.

(C)  Support full multimedia functions. Enable users to control, manipulate and transform different multimedia data and information.

(2)   A visual programming environment.

(A)  Enable user to design and implement complete applications without the need to write any textual code.

(B)  Scalability and cross platform compatibility.

2.2     C++ vs. Java

NeatTools is an experimental computer science project. Before we can undertake any experiment, we need to have a tool or an environment. This makes it possible to build and test design concepts and proceed with the experiment itself. So, the first step was to implement such a visual programming tool. The question was which language to use to implement the tool so it would meet all the requirements and still have the potential for future development. C/C++ is the industry standard and Java is a rapidly developing and widely adopted new language. This section discusses the pros and cons of these two popular languages.

2.2.1     Java Background

Java is an object-oriented programming language developed by Sun Microsystems, a company best known for its high-end UNIX workstations. Modeled after C++, the Java language was designed to be small, simple, and portable across platforms and operating systems, both at the source and at the binary level.

Platform independence is one of the significant advantages that Java has over other programming languages. At the source level, Java's primitive data types have consistent sizes across all development platforms. Java's foundation class libraries make it easy to write code that can be moved from platform to platform without the need to rewrite it to work with another platform. Java binary files are also platform-independent and can run on multiple platforms without the need to recompile the source. How does this work? Java binary files are actually in a form called bytecodes. Unlike most other programming languages, The Java development environment has two parts: a Java compiler and a Java interpreter. The Java compiler takes the Java program and instead of generating machine code from source files, it generates bytecodes. To run a Java program, a user runs a program acting as a bytecode interpreter, which in turn executes a Java program. The user can either run the interpreter by itself, or for applets, there is a bytecode interpreter built into a Java-capable or enabled browser like Netscape [29] or Microsoft Internet Explorer [8] which runs the applet.

The disadvantage of using bytecodes is in execution speed. Because system-specific programs run directly on the hardware for which they are compiled, they run significantly faster than Java bytecodes which must be processed by the Java interpreter. In order to increase the performance of Java bytes, some companies are dedicated to accelerating the bytecode. JIT [25] (just in time) interpreter is one of the technologies that is widely adapted into commercial Web browsers. JIT interpreter usually loads the Java bytecodes, preprocesses it into system-specific code in a dynamical fashion, and then executes the system-specific code directly.

2.2.2     Old Benchmark

I wrote a simple Java benchmark program:

import java.applet.Applet;

 

public class benchmark extends Applet {

  public int dummy(int i) { return i;}

  public void start() {

    int i, x;

    x = 0;

    System.out.println("Benchmark Start");

    long b1 = System.currentTimeMillis(), b2;

    for (i=0; i<100000000; i++) {

    }

    b2 = System.currentTimeMillis();

    System.out.println("Empty For Loop "+(b2-b1));

    b1 = b2;

    for (i=0; i<100000000; i++) {

      x += i;

    }

    b2 = System.currentTimeMillis();

    System.out.println("For Loop with + and assign "+(b2-b1));

    b1 = b2;

    for (i=0; i<100000000; i++) {

      x *= i;

    }

    b2 = System.currentTimeMillis();

    System.out.println("For Loop with * and assign "+(b2-b1));

    b1 = b2;

    for (i=0; i<100000000; i++) {

      x += dummy(i);

    }

    b2 = System.currentTimeMillis();

    System.out.println("For Loop with function call and assign "+(b2-b1));

  }

}

 

Translating the Java program into C++ would be:

#include <stdio.h>

#include <sys/timeb.h>

 

long currentTimeMillis() {

  struct _timeb tstruct;

  _ftime(&tstruct);

  return tstruct.time*1000+tstruct.millitm;

}

 

class benchmark {

  public:

    int dummy(int i) { return i;}

    void start() {

      int i, x;

      x = 0;

     

      printf("Benchmark Start\n");

      long b1 = currentTimeMillis(), b2;

      for (i=0; i<100000000; i++) {

      }

      b2 = currentTimeMillis();

      printf("Empty For Loop %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x += i;

      }

      b2 = currentTimeMillis();

      printf("For Loop with + and assign %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x *= i;

      }

      b2 = currentTimeMillis();

      printf("For Loop with * and assign %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x += dummy(i);

      }

      b2 = currentTimeMillis();

      printf("For Loop with function call and assign %d\n",

        b2-b1);

    }

};

 

void main() {

  benchmark bm;

  bm.start();

}

 

The test data are below (Test on PC with AMD-233MHZ CPU[30], Win95) :

Units (ms)

for loop 108

x += i

x *= i

x += dummy(i)

C++, VC++

0 *

880

1810

880

JDK 1.1 [25]

29000

50420

51300

110730

JDK 1.0

43830

82340

83210

170210

IE 4.0

42400

63330

63820

175550

IE 4.0 (JIT)

0*

930

2250

9010

Netscape 4.05

940

1810

2690

8510

Table 1: Old benchmark (1)

* Optimized and eliminated by compiler or interpreter.

Units (us/loop)

for loop

x += i

x *= I

x += dummy(i)

C++, VC++

0 *

0.0088

0.0181

0.0088

JDK 1.1

0.29

0.5042

0.513

1.1073

JDK 1.0

0.4383

0.8234

0.8321

1.7021

IE 4.0

0.424

0.6333

0.6382

1.7555

IE 4.0 (JIT)

0*

0.0093

0.0225

0.0901

Netscape 4.05

0.0094

0.0181

0.0269

0.0851

Table 2: Old benchmark (2)

* Optimized and eliminated by compiler or interpreter.

2.2.3     New Benchmark

The old benchmark has a major problem – it does not reflect the correct timing on the C++ function call. The compiler optimizes away the code sections, which handle the function jump and return related operations. In C++ object oriented design, the virtual function play an important role in the object inheritance and polymorphism. Because virtual function is dynamically linked in run time, compiler could not remove the actual function call operations. Thus, we could modify our old benchmark C++ code and make the comparison with Java more sensible. (In Java, all functions are virtual functions, so we don’t have to worry about this problem.)

The new C++ benchmark problem become:

#include <stdio.h>

#include <sys/timeb.h>

 

long currentTimeMillis() {

  struct _timeb tstruct;

  _ftime(&tstruct);

  return tstruct.time*1000+tstruct.millitm;

}

 

class benchmark {

  public:

    virtual int dummy(int i) { return i;}

    void start() {

      int i, x;

      x = 0;

     

      printf("Benchmark Start\n");

      long b1 = currentTimeMillis(), b2;

      for (i=0; i<100000000; i++) {

      }

      b2 = currentTimeMillis();

      printf("Empty For Loop %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x += i;

      }

      b2 = currentTimeMillis();

      printf("For Loop with + and assign %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x *= i;

      }

      b2 = currentTimeMillis();

      printf("For Loop with * and assign %d\n", b2-b1);

      b1 = b2 = currentTimeMillis();

      for (i=0; i<100000000; i++) {

        x += dummy(i);

      }

      b2 = currentTimeMillis();

      printf("For Loop with function call and assign %d\n",

        b2-b1);

    }

};

 

void main() {

  benchmark bm;

  bm.start();

}

 

The test data are below (Test on PC with Intel-400MHZ PII CPU, Win98) :

Units (ms)

for loop 108

x += i

x *= i

x += dummy(i)

C++, VC++

0 *

940

1200

2970

JDK 1.0.2

36300

74320

72280

138690

JDK 1.1.8(JIT)

1260

1370

1380

4390

JDK 1.2.2(JIT)

1270

1430

1310

4450

IE 5.1(JIT)

600

940

1480

3240

Netscape 4.7

550

1810

1870

3290

Table 3: New benchmark (1)

* Optimized and eliminated by compiler or interpreter.

Units (us/loop)

for loop

x += i

x *= i

x += dummy(i)

C++, VC++

0 *

0.0094

0.012

0.0297

JDK 1.0.2

0.363

0.7432

0.7228

1.3869

JDK 1.1.8(JIT)

0.0126

0.0137

0.0138

0.0439

JDK 1.2.2(JIT)

0.0127

0.0143

0.0131

0.0445

IE 5.1 (JIT)

0.006

0.0094

0.0148

0.0324

Netscape 4.7

0.0055

0.0181

0.0187

0.0329

Table 4: New benchmark (2)

* Optimized and eliminated by compiler or interpreter.

2.2.4     Benchmark Analysis

Based on the nature of object oriented programming, a function call is a must. Class interfaces are actually functions associated with objects. From the table above, we learn that the most advanced commercial browser's Java interpreter like IE5.1 and Netscape 4.7 is about only 1.05 times slower than compiled C++ code when a virtual function call is involved in the loop. When the Java interpreter is not equipped with JIT, the ratio becomes unacceptably slower (around 46.7 times slower). During these years, JIT become a mature technology, it is now a necessary portion of most JVM implementations. This benchmark is only a very simple benchmark program. When it comes to large applications in the real world, the JIT tends to have worse performance because it has a limited buffer for compiled code. If the whole application is too large to be compiled before it executes, the JIT interpreter will have to compile section by section on the fly and will thus slow down the execution speed. I estimate the JIT would run about 2 times slower than compiled C++ code when an application is relatively large. But, Java is changing and evolving rapidly compared to C/C++; JDK itself has gone through several changes and enhancements including JDK 1.1, JDK 1.2 [25], JDBC [25], Java 3D [25], JIT; and the long awaited Java compiler, HotSpot [25]. A lot of experts in this field believe that it is possible that Java could increase in speed, performance, and eventually exceed what C/C++ does in the future. This means that the benchmark result today may not be similar in the future, but it could state the facts from the past until now.

2.2.5     Conclusion

When I started to design the next generation of the NeatTools project, I insisted on using C++ instead of Java. The main reason for that is speed and Java's uncertainty in the future. At that time, the JIT technology was not mature. Even today the JIT Java interpreter is still not fast enough for mission critical and calculation intensive real-time tasks like compress/decompress of voice and video data. However, like I discussed before, the benchmark result could only reflect Java as an immature technology at this time. It might not always be true in the future. Today, there are some other solutions such as linking native code into Java programs or using tools to convert Java bytecode into native code. But these alternatives are still not very efficient or stable. With the urgent need to help disabled people and solve other problems in the NeatTools project, my choice to use C++ was appropriate, because, I could then focus on the unique features and strategies for specific applications and experiments in NeatTools, not the language issue itself. In the future, when Java does become superior to C/C++, we may choose to rewrite NeatTools and adapt all the experience we have acquired towards implementing them in Java.

2.3     How to get JAVA without using Java

Here I would like to discuss some architecture issues. First, I will list the C++ features that Java avoids and investigate why they might be favorable. Actually, a programmer could benefit from those features when those features are well understood and used properly. Secondly, some features that Java adds could be useful. I will discuss how to implement the C++ counterpart and provide the desired functionality to the programmer.

2.3.1     C++ Features that Java avoids

2.3.1.1     Multiple Inheritance

Java provides single inheritance that is a subset of C++ multiple inheritance. A Java class "extends" one super class and can "implement" multiple interfaces (abstract classes without any implementation and non-static data members). In C++, a class can inherit from a set of classes including abstract classes (classes with one or more functions declared as abstract) or regular classes.

In theory, Java could provide the functionality of multiple inheritance by implements multiple interface classes. But in practice it could lead to code reusability problems. Lets look at the following example.

In Java 1.1, the new API provides a new mechanism that could make event broadcasting and listening more natural. This is a good use of object oriented programming. In the AWT class, the user usually extends from Canvas or Panel class to provide a windowing class with specialized functionality. In order to provide the basic action event broadcasting, the user usually has to implement the following methods like addActionListener(), removeActionListener(), and broadcastEvent(). The code is as follows:

ActionListener actionListener = null;

 

public void addActionListener(ActionListener l) {

  actionListener = AWTEventMulticaster.add(actionListener, l);

}

   

public void removeActionListener(ActionListener l) {

  actionListener = AWTEventMulticaster.remove(actionListener, l);

}

   

public void broadcastEvent(ActionEvent e) {

  if (actionListener != null)

    actionListener.actionPerformed(e);

}

 

The code above is simple and straightforward. Because not every windowing class needs to dispatch action events, these methods are not provided with the Component class (the super class for every windowing class). Now, there is no way for the Java programmer to reuse the code in different classes with different inheritance paths because of Java's single inheritance. For example, class A and class B are derived from Component and both classes need to broadcast action events. Then, both classes need to implement the exact same code segment and could not just inherit it from a separate interface (the interface does not carry implementation, only method declarations).

By using a different design pattern, one may design a separate class ActionBroadcast that can handle the event broadcast and include the class as a data member. That way we can wrap the methods, and the member class variable provides the actual implementation.

ActionBroadcast broadcast = new ActionBroadcast();

 

public void addActionListener(ActionListener l) {

  broadcast.addActionListener(l);

}

   

public void removeActionListener(ActionListener l) {

  broadcast.remvoeActionListener(l);

}

   

public void broadcastEvent(ActionEvent e) {

  broadcast.broadcastEvent(e);

}

 

This solution solves the code reusable problem and, in the mean time, leads to an inefficient coding style. Because the method wrapper is always slower than a direct function call. When the underlying implementation is complex, the wrapper pattern will not account for too much performance loss. For the example above, this approach does not provide any benefit in terms of code reusability or efficiency.

2.3.1.2     Operator Overloading

In Java, operator overloading is eliminated completely. Though, the effects of operator overloading can be achieved by declaring a class, appropriate instance variables, and appropriate methods to manipulate those variables. I believe good use of operator overloading could actually increase the readability and sometimes the efficiency of the program.

For example, in some arithmetic classes like Vector, Matrix, and Coordinate, the use of operator overloading like addition, subtraction, multiplication, and division are very natural and could lead to a much more concise syntax.  Without operator overloading, some complex calculations could become ugly and almost unreadable. Sometimes, one line of program expressed by variables and operators has to rewrite into several lines when expressed by variables and methods. This situation could cause the compiler to generate less optimized code and decrease efficiency.

Let's say we have Point3D as a 3D coordinate, and we want to calculate the point rotation with angle alpha. In Java, we will need to define the mul(double) add(Point3D) methods by the following statement:

Point3D p1 = new Point3D(1, 0, 0);

Point3D p2 = new Point3D(0, 1, 0);

p1.mul(Math.sin(alpha)).add(p2.mul(Math.cos(alpha)));

 

In C++, we define the operator*(double) and operator+(Point3D) methods and the statement becomes:

Point3D p1(1, 0, 0);

Point3D p2(0, 1, 0);

p1*sin(alpha)+p2*cos(alpha);

 

As you may notice, in C++, the statement is much shorter and easier to understand. Also, the notion of operators is natural to how we express mathematic equations. For complex Matrix or Vector operations, the statement in Java could become very long and unreadable. I believe, for a project group or for someone else to read the code, readability is a very important factor. In C++, we have the flexibility accomplish this by operator overloading or just by methods.

2.3.1.3     Pointer

The main reason Java eliminates pointer is that pointers are one of the primary features that enable programmers to inject bugs into their code. But, to my understanding, object oriented design could avoid class user's pointer usage completely. Also, good use of pointer could increase the efficiency and simplify the code generation.

In Java, the only way that a method could return primitive data type values in its parameters is through the use of Array.

int arg[] = new int[1];

foo(arg);

 

In C++, the same code could be expressed as:

int arg;

foo(&arg);

 

Obviously, the implementation in C++ is much better then Java. In Java, it requires more memory space (both reference for arg reference and int[] array class instance) and also it needs to implicitly initialize the internal Array class instance (which has unavoidable error checking code in Java API) for "arg" as an array of integers.

For those situations that pointer exposure is unwanted, we could use object class to hide the pointer and, in the mean time, provide the functionality needed. For example, to avoid user handle pointer directly, we could design IntArray class:

class IntArray {

    int *data;

  public:

    IntArray(int size);

    ~IntArray();

    int operator[](int index);

};

 

The IntArray class could encapsulate the integer pointer inside the class. It keeps the class user from using the pointer directly. In the mean time, it could allocate and de-allocate the memory properly in constructor and destructor. So, we don't have to worry about the pointer usage and memory allocation all together.

2.3.1.4     Memory Allocation

In the "Java white paper" [25], it states:

"Java completely removes the memory management load from the programmer. C-style pointers, pointer arithmetic, malloc, and free do not exist. Automatic garbage collection is an integral part of Java and its run-time system. While Java has a new operator to allocate memory for objects, there is no explicit free function. Once you have allocated an object, the run-time system keeps track of the object's status and automatically reclaims memory when objects are no longer in use, freeing memory for future use."

In theory, humans can always design better algorithms in compiler or interpreter to optimize the performance of high-level language. But Assembly language is still there to maximize the performance for some mission critical tasks in specialized computer industry. The most advanced-programming wizard could create quite a functional skeleton source code, but when it comes to specific requirements, an experienced programmer could still do a better job. Here, I try to express the idea that the human factor could never be eliminated in the decision making process. In the memory management case, according the different requirements and constraints, we can decide to implement a garbage collector in C++ or use object oriented programming to hide the memory management inside an object and distribute the memory management efforts into the object hierarchy. The programmer could use the class and never have to deal with the memory management problems directly. We can also decide to implement a reference count to eliminate unnecessary memory duplications, speed up the performance, and minimize the space requirement. In short, with proper set of classes support, carefully programmed C++ is robust and free of memory leak. There are a great deal of different algorithms and implementation that we can choose in C++ or other object oriented programming languages that provide memory management primitives, but in Java we lose the right to choose. In C language's design spirits, it trusts programmers that they could do their job right. In Java, it only trusts the system designer. When a system designer fails to make good implementations, Java programmers have no choice but to use it anyway. For example, in Java3D, the API itself is still in beta test and has several bugs. One of the most complained about issues is that it consumes too much memory even when users only construct a fairly simply scene. I believe part of the reason lies in Java's memory management implementation. The garbage collector could automatically reclaim the unused memory. But the programmer's convenience comes with a price - we could nullify a class reference right away, but we don't know when it will be collected and reclaimed. For those big chunks of memory blocks (textual map, image buffer, or z buffer), they need to be claimed and reclaimed in the right order and in the right time. In this case, the garbage collector can not perform well and usually will cause memory usage problems.

2.3.1.5     Fragile Super Class

Fragile super class problems happen when a programmer modifies the header file of a base class; all the derived class which use this header file will need to be recompiled, or the executable linkage problem could occur. I believe this problem is inherited in the implementation of the current C/C++ compiler, not in the language itself. C/C++ compiler tends to parse the method name and data member identifier into an internal address. When the structure or the order of the identifier changes in the header file, the internal address and the lookup table change also. Without re-compiling the derived class's source code, the internal address of the methods or data members will not match. Thus, the fragile super class problem occurs - the programmer has to re-compile the super class before it could be linked to the derived classes. With a proper naming lookup protocol, this problem could be solved. On the other hand, a better way to solve the problem is to avoid changing the super class's interface. I believe, before the coding draft, a detailed class design and analysis will ensure a proper interface of a super class. Thus, we could avoid modifying the base class header file and prevent the fragile super class problem from happening.

2.3.2     Some Features that Java Adds

2.3.2.1     Build-in Thread and Synchronization Primitives

By using object oriented programming, multithreading capabilities could be added in C++ easily. The usage could be similar to what Java has. In the Thread class, we can have a class that uses Thread primitive (In Microsoft Windows, the Win32 API already has built-in thread support. In most UNIX systems, the pThread [31] is an add-on package to support the POSIX [31] thread) to create a new Thread and store its handle in the data member of Thread class. Later on when the thread is ready to start, we can redirect the thread process flow to the run() method. Thus, when user derives class from Thread class or Runnable class, it could then override the run() method and implement its thread loop. In "Java Like Cross Platform API," I follow the Java API standard and implement the JThread class in C++. It was proved that the API could hide the platform dependent implementation nicely, and programmers could have multithreading capabilities without noticing its underlying implementation differences.

To deal with concurrent problems, Java defines a new method modifier to integrate the thread synchronization into language. I believe this is a good move. But in C++, we can implement the mutex and critical section control into utility classes to help us handle the thread synchronization problems. In this case, we could have flexibility with how we want our process flow to be guarded by those synchronization primitives and make suitable adjustments to adapt to the needs of different situations.

2.3.2.2     Build-in Array

In Java, the Array class was created implicitly. The "new Data_Type[size]" syntax allows the user to create Array of any data type. It is very convenient, as there is no need to make type casting for the returned index elements. In the old-fashioned C++ design, the only way to make an Array is using the pointer of object pointer to implement the object array:

objectArray {

    int length;

    Object **data;

  public:

    ObjectArray(int size);

    ~ObjectArray();

    Object* operator[](int index);

};

 

Here, the polymorphicism in C++ allows us to use ObjectArray to store any class that is derived from Object class. But the element returned by operator[] could only return a pointer of Object which needs to be cast before it could be used properly. Also, it could not store primitive data type and other classes which do not derive from Object class. So, in practice, it is not a good solution for a universal array.

Fortunately, C++ now includes a "template" which could embed the data type as a parameter into the class declaration and solve the problem for an easy usage universal array. The template could be declared as follows:

template <class DataType>

objectArray {

    int length;

    DataType *data;

  public:

    ObjectArray(int size);

    ~ObjectArray();

    DataType& operator[](int index);

};

 

Now, the same template could apply to any class and primitive data type as well. Also the elements return by operator[] will return the desired data type. So, the use of template could provide exactly the same functionality that Java's build-in array provides.

2.3.2.3     Cross platform and Windowing API

In C++, the windowing related task is never easy. I remember, in the early books that discussion of how to open a simple window and display "Hello World!" in any Window system was fairly complex. The program was usually more than 100 lines. Since C++/C was designed for environments with only text input/output, windowing was not natural and straightforward.

Later on, as the windowing environments became more and more popular, most of the C/C++ compilers began to provide the IDE (Integrated Development Environment) with graphical user interfaces running on a windowing environment. Also, compilers came with API (like OWL [32], MFC [8], and MOTIF [33]) that assist the programmer in creating their own graphical user interfaces. The problem is APIs are very different from one another. Also, for different windowing systems, there are all sorts of different APIs.

As the Internet becomes more prevalent, the interaction between different hosts running different operating systems and locations becomes more and more common and hence more and more important. Java was the first language, which supported cross platform development on the language level. Java's virtual machine and byte code system is the natural result of the language specification to be a cross platform language. I have discussed the cons and pros of this design. My solution to cross platform for C++ is an API that provides uniform support to basic utility and abstract windowing operations. It could encapsulate the underlying implementation, which is based on the windowing environment. So application dependency to windowing environment is eliminated, and the goal of cross platform in C++ could be easily achieved.

2.3.3     Conclusion

In this section, I examined the C++ features that Java avoids and discuss how to avoid the potential pitfalls. Instead of eliminating them, we could try to understand them, making good use of them and benefiting from those features. Overall, C++ is more flexible than Java. It provides multiple inheritance, operator overloading, and pointer. It could add code reusability, readability, and efficiency. For those useful features that Java adds, I propose a practical way to provide them in the C++. So C++ programmers could get Java's features without using Java. My conclusion is: object oriented programming in C++ is so powerful that it could be used to avoid commonly seen problems and also it could be extended with new kinds of functionality as well.

2.4     NeatTools Architecture Analysis

In NeatTools, the most important concept is module abstraction. The module abstraction simplifies and models all the interactions between modules into active actions and reactive actions. Later in this section, I will discuss the implementation concept and present some benchmark information and analyze it.

2.4.1     Module Abstraction


Figure 1: Module Abstraction

In NeatTools, module abstraction is offered as a set of class methods that provide inter-module communication functionality. Functional components (implemented as objects) of a concurrent system are written as encapsulated modules that act upon local data structures, or objects inside object classes, some of which may be broadcast for external use. Relationships among modules are specified by logical connections among their broadcast data structures. Whenever a module updates data and wishes to broadcast the change, and make it visible to other connected modules, it should call an output service function which will broadcast the target data structure according to the configuration of logical connections. Upon receiving the message event, the connected modules execute an action engine according to the remote data structure. Thus, output is essentially a byproduct of computation, and input is handled passively, treated as an instigator of computation.

This approach simplifies module programming by cleanly separating computation from communication. Software modules written using module abstraction do not establish or effect communication, but instead are concerned only with the details of local computation. Communication is declared separately as logical relationships among the state components of different modules.

This programming model has its roots in the formal Input / Output automaton model of Lynch and Tuttle [1]. An I/O automaton is a state machine with a signature consisting of a set of input actions and a set of locally controlled actions. The locally controlled actions could divide into output actions and internal actions. Locally controlled actions are under the control of the automaton, while input actions may occur at any time. Automata may be composed such that when an output action of one automaton occurs, all automata having a same-named action as an input action may make a state transition simultaneously. A behavior of an I/O automaton is a sequence of input and output actions that may occur in an execution of that automaton. The module abstraction programming model is designed to benefit from the useful characteristics of the I/O automaton model that are helpful in reasoning formally about distributed systems.

Module abstraction is based on three fundamental concepts: data, actions, and connections. It is difficult to discuss these concepts in detail without reference to particular mechanisms for supporting them. Therefore, we present them in the context of NeatTools, a software run-time system and visual programming environment designed to support the development of data flow network applications using module abstraction.

2.4.1.1     Data

Data (the components of a module's state, could be data structures or objects) may be kept private, or they may be broadcast when needed so that other modules may access the data. NeatTools provides a base abstract object that declares the basic data structure and service procedures (or methods). Every module object should inherit from this object and override some predefined procedures to serve the different computational and presentational needs of each module. NeatTools provides a library of data types for declaring data that may be broadcast. These include but are not limited to integer, real, string, block, byte array, midi event, voice stream, and video stream. The module programmer may define others.

2.4.1.1.1     The Presentation

Each NeatTools module has a presentation that presents itself to the user as visual feedback. It could be graphical images, shapes, or text. The presentation may change dynamically according to the current state. Associated with each data item are a public name, property, access privileges, and data type. This information helps a user understand its presentation. The data type and access privileges also permit type check and privilege match of logical connections.

2.4.1.1.2     Access Privilege

Access privileges include input, output, insert, and connect. Output access allows connected modules to observe the value of the data when broadcast as an event message. Input access allows a module to change the state of the target module. Insert access allows a new connection to be inserted into a module. Connect access allows a module to relate the data item to a data item of some other module. All those access privileges are controlled by a set of class methods (already defined in the base abstract class). By overriding those methods, modules could change the behavior or dynamically control access behavior according to the current state of the module.

2.4.1.1.3     The Desktop

A NeatTools module interacts with a desktop and a collection of other modules that may be unknown to this module but that read and modify the data item in its presentation when permitted by the access privileges. The desktop also works as a graphical user interface front end that provides a user with a set of layout service functions, including move, resize, copy, delete, group, ungroup, connection management, object persistency management and file input/output.

2.4.1.2     Actions

The action portion of a module defines how its state changes over time or responds to an environment. Insulated from the structure of its environment, a NeatTools module interacts entirely through the broadcast service and reactive execution engine procedures. A module may autonomously modify its local state, and also it may react to the incoming events and then change its local state. This suggests a natural division of the actions into two parts: active action and reactive action.

2.4.1.2.1     Active (Output) Action

The active action carries out the ongoing computation of the module. For example, in a discrete event simulation, the active action would be iterative computation that simulates each event. External updates of simulation parameters could affect the course of future iterations, but would not require any special activity at the time of each change. Modules with only active action can be quite elegant since input simply steers the active computation without requiring a direct response. Active action is analogous to the locally controlled actions of an I/O automaton.

2.4.1.2.2     Reactive (Input) Action

The reactive action carries out activities in response to input from the environment. A module with primarily reactive action simply reacts to input from the environment, updating its local state and presentation as dictated by the input change. For example, a data visualization module could be constructed so that each time some data element changes, the visualization is updated to reflect the change. In the above discrete event simulation, one might add reactive action to check the consistency of simulation parameters that are modified by the environment. Reactive action is analogous to the input actions of an I/O automaton.

2.4.1.3     Connections


Figure 2: Connection

Relationships between data items of different modules are declared with logical connections between those data items. These connections define the communication pattern of the system. Connections are established by a special NeatTools module, called a desktop, that enforces type compatibility across connections and guards against access protection violations by establishing only authorized connections.

Connections are declared separately from modules so that one can design each module with a local orientation and later connect them together in various ways. Connections are designed to accommodate all kinds of data types varying from integer and real number to audio and video. The run-time system could handle the communication requirements automatically according to the module abstraction of the module.

If we liken the data items of a NeatTools module to the actions in the signature of an I/O automaton, then just as like-named actions in automaton signatures define the sharing of actions, connections define the sharing of state change information. Currently, a simple synchronous data transmission algorithm is used by the broadcast service procedure. The reactive action engine of a connected module will be invoked and executed automatically. However, if asynchronous data transmission is needed, a user could construct a data queue inside the module and react to the data queue later. This way, we can keep the general communication structure simple and efficient.

2.4.2     Implementation Concept


Figure 3: Implementation Concept (1)


Figure 4: Implementation Concept (2)

The NeatTools' design concept is based on a special aspect of application design, which provides visual programming capacity. In traditional textual programming design, lines of code are the rough measure of program size. In a visual programming design like NeatTools' data flow networks, the number of connections and modules become the major indicators. Thus, how NeatTools provides efficient way to broadcast and process message events between modules becomes the most important issue. In NeatTools, all modules are already objects. The only object interoperability is simplified into active and reactive actions, or connections. In order to increase the application throughput, the connections in NeatTools are just logical references. The message broadcasts are actually performed by direct function calls to ensure the performance of NeatTools' data flow network. Remote messages passing between NeatTools and a remote computer running NeatTools are performed by Socket and ServerSocket modules (just like regular modules, but they perform different tasks) to handle all the remote data communication tasks (usually on top of TCP/IP which is slow compared to a direct function call). Connections between Socket and other modules still use the same efficient function call. So, instead of a network connection oriented design (like CORBA [20]), NeatTools uses a direct function call oriented design that eliminates possible overhead and layers. Also, the function call oriented design guarantee the message will never disappear. In C++, when a function is called, it will always return after it finish execution except there are unprocessed exceptions or system interrupts that force the program to abort. Thus, the message broadcast is always "synchronous". About the concurrent process and underlying mechanism is discussed in appendix (module programming introduction).

2.4.3     Benchmark Information on NeatTools

I wrote a special benchmark module in NeatTools, which has two inputs and one output. The one on top can be enabled to start the benchmark. The one on the left can receive message events from other benchmark modules but do nothing. When the module is enabled, it will start the benchmark by broadcasting events through its output port in a 106 loop and then display the time interval in the debug dialog box. The benchmark here is focused on the overhead of message broadcasting in NeatTools.

The engine method of benchmark module was implemented as follows:

void JBenchmarkObj::engine(int n, JLinkObj& link) {

  if (!n) {

//    link.access(JIntegerData(v[0]));

  } else {

    int oldv = v[1];

    link.access(JIntegerData(v[1]));

    v[1] = (v[1] != 0);

    if ((v[1] != oldv) && !v[1]) {

      int time = JSystem::currentTimeMillis();

      for (int i=0; i<1000000; i++)

        broadcast(0);

      time = JSystem::currentTimeMillis()-time;

      JComponent::debug(JInteger::toJString(time)+" ms");

    }

  }

}

The NeatTools benchmark data flow networks are arranged as follows:

Figure 5: Speed Test

The five test modules are instances of a benchmark module. One of them connects its enabled input from a button, and its output connects to the rest of the test modules.

2.4.4     Benchmark Analysis of NeatTools

The benchmark data for this particular data flow network running on my PC (AMD-233MHZ CPU, Win95) is 2200 ms, because the output has four connections and the broadcast processes are repeated for 106 times. So 2200 ms / 4 / 106 becomes 0.55 us per connection broadcast which is just 6.25 times slower compared to the direct function call with only one statement. In other words, NeatTools is capable of  dispatching around 1,800,000 messages in one second. At this speed, I believe NeatTools could handle most real-time or computationally intensive tasks on the data flow network level.

2.4.5     Conclusion

The purpose of this section was to focus on how I proposed an abstract module model. Based on this model, I introduced a simplified and high performance way to implement the module connections, and how to map the connections into direct function call without much overhead or layers. This model proved successful as implemented in the NeatTools project. With the scalable design of the NeatTools project, I believe NeatTools could be widely adapted to most real world problems and provide complete and fast solutions.

2.5     NeatTools and Object Management Environments

2.5.1     CORBA


Figure 6: CORBA

Object Managements Group [20] (OMG)'s Object Management Architecture (OMA) is the multi-vendor standard for object-oriented distributed computing. This includes CORBA -- the Common Object Request Broker Architecture -- which most people associate with OMG; CORBAservices and CORBAfacilities.

2.5.1.1     Object Definition

Objects are discrete software components -- they contain data, and can manipulate it. Usually, they model real-world objects, although sometimes it's useful to create objects specifically for items we want to compute. Other software components send messages to objects with requests; the objects send other messages back with their responses.

2.5.1.2     CORBA Provided Object Interoperability

In order for objects to plug and play together in a useful way, clients have to know exactly what they can expect from every object they might call upon for a service. In CORBA, the services that an object provides are expressed in a contract that serves as the interface between it and the rest of our system. The interfaces are expressed in OMG Interface Definition Language -- OMG IDL -- making them accessible to objects written in virtually any programming language, and the cross-platform communications architecture is the Common Object Request Broker Architecture -- CORBA.

2.5.1.3     OMA Provided Application-Level Integration

Based on CORBA architecture, the OMA specifies a set of standard interfaces and functions for each component. Different vendors' implementations of the interfaces and their functionality then plug-and-play on customers' networks, allowing integration of additional functionality from purchased modules or in-house development.

The OMA is divided into two major components: lower-level CORBAservices and intermediate-level CORBAfacilities. The CORBAservices provide basic functionality that almost any object might need: object lifecycle services such as move and copy, naming and directory services, and other basics. The CORBAfacilities architecture has two major components: one, horizontal, including facilities such as compound document services which can be used by virtually every business; and the other, vertical, standardizing management of information specialized to particular industry groups.

2.5.2     COM/DCOM


Figure 7: COM/DCOM

COM [8] (Component Object Model) is a software architecture that enables a program to be built from smaller binary components. It is a binary standard for component interoperability and is independent of any programming language.

COM supports a client/server model between the user of some object's services and the implementers of that object and its services. COM's role is to establish the connection between the client and the server, which offers the desired object. Once the connection has been made, COM is out of the picture and all communication goes directly from server to client and vice versa.

2.5.2.1     Objects and Interfaces

In COM an object is an instance of a class which as in standard OO terminology is a set of data and related functionality. Unlike most other OO models, COM provides no direct access to object data. Instead, users have to access member functions of an associated interface.

An interface is a set of functions that can be invoked on a given object. Interfaces do not contain any implementation whatsoever, but merely define the expected behavior of an object. When an object implements an interface, it provides implementations for every function in that interface, and provides pointers of those functions to COM.

2.5.2.2     Interface Description Language (IDL)

COM's Interface Description Language (IDL) is based on the Open Software Foundation (OSF) [33] Distributed Computing Environment (DCE) [33] specification for describing interfaces, operations, and attributes to define remote procedure calls.

A designer can define a new custom interface by writing an interface definition file. The interface definition file uses the IDL to describe data types and member functions of an interface. The interface definition file contains the information that defines the actual contract between the client application and the server object.

2.5.2.3     Service Control Manager (SCM)

The SCM is a COM component that is able to locate a given server and launch it. The SCM contains a database of class information. When a client requests the COM library to create an object, the SCM is launched, the server located and run. Here SCM provides the object interoperability.

2.5.3     A Comparison of CORBA and COM/DCOM

From the communication point of view, COM is a little bit different from CORBA. In CORBA, ORB is always the gateway between its object client and server. The client and server in CORBA never communicate with each other directly. In COM, SCM locates and launches the object server for the application client, and then the client communicates directly with the server. Also COM has different mechanisms for in-process, cross-process, and remote object servers. In in-process cases, the SCM will locate and load the object server as a DLL. So in this situation, the object server will be executed in the client's process space, which is much faster than CORBA's mechanism. In cross-process cases, the SCM will locate and load the object server as an executable. The object server will be executed in a separate process space. So it is slower than the DLL but still faster then the CORBA's mechanism. Even in remote object server cases, the SCM will contact the remote SCM and later on build a remote proxy server which can forward requests directly to the remote SCM via the RPC connection. So, in general, COM/DCOM has better performance than CORBA. But CORBA has better design based on system and structured concerns. For example, a CORBA's object server will only deal with its local ORB. But a DCOM's object will have to deal with local SCM directly, or in-process client, cross-process client, and remote client through an interface wrapper.

From the application point of view, COM is lacking in application integration packages. CORBA's CORBAservices and CORBAfacilities components provide application-level integration which in turn provides lower-level and intermediate-level services for industry and business applications.

2.5.4     Benchmark Information on CORBA

The most important advantage that Object Management Environments like CORBA, and COM/DCOM is object interoperability. Object interoperability allows objects, which plug and play together in a useful way to become possible. But the advantage comes with a price -- speed. For example, in CORBA, when an object client issues a function call to a remote server object, it has to go through the IDL Stub (created by IDL), Object Request Broker (ORB), remote ORB, IDL Skeleton, and finally reaches the remote server object implementation. And in most cases it goes through a network communication layer like TCP/IP at least twice (forward the request and send back the result message) to complete a function call.

I visited one of the ORB vendor sites which has a web page dedicated to performance information. The URL for this page is http://www.orl.co.uk/omniORB/omniORBPerformance.html. From this page, came one of the tables below:

Performance of omniORB2 on various platforms.

Platform

Transport

Protocol

us/call

LINUX Pentium Pro 200 MHz

IP/intra-machine

IIOP

340

 

IP/ethernet (ISA)

IIOP

1000

 

IP/ATM

IIOP

440

 

AAL5/ATM

IIOP

350

Windows NT Pentium Pro 200 MHz

IP/intra-machine

IIOP

360

 

IP/ethernet (ISA)

IIOP

1000

Digital UNIX 3.2 Alpha DEC 3000

IP/intra-machine

IIOP

750

 

IP/ethernet

IIOP

1050

Windows '95 Pentium 166 MHz

IP/intr-machine

IIOP

1000

 

IP/ethernet (PCI)

IIOP

1250

SOLARIS 2.51 Ultra 1 167 MHz

IP/intra-machine

IIOP

540

 

IP/ethernet

IIOP

710

Table 5: CORBA benchmark infomation

2.5.5     Benchmark Analysis of CORBA

From the table above, to make a function call, even if the function call is made within the machine (the IP/intr-machine), it takes about 360 us on a Pentium Pro 200MHz machine. If we compare the benchmark information with the previous section, making a direct function call in compiled C++ code only takes 0.0088 us which is about 41,000 times faster than making function calls on this particular ORB. For those calls that are made remotely by a direct network card connection (the IP/ethernet), it takes 1,000 us to make a function call. In that case it is about 114,000 times slower than the compiled C++ code.

2.5.6     Conclusion

CORBA and COM/DCOM provide object interoperability, which is a great feature for object plug and play and object reusability. In CORBA, it especially tries very hard to separate the object server from object client to ensure encapsulation and increases the object interoperability by using ORBs. But this approach comes with a big draw back -- decreased performance to a great degree. In COM/DCOM, the performance problem is less severe. Because COM/DCOM provides direct access to the object server when an object server is loaded as an in-process DLL. But, even though COM/DCOM is independent of the programming language, some of the higher level control mechanisms involve registry operations in Microsoft Windows like Win95 or NT which we try to avoid in NeatTools due to cross platform concerns.

It is possible that for the NeatTools project to be built upon CORBA architecture. A NeatTools module designer will be able to put the modules on line and allow user remote access. But, NeatTools will have to limit data flow design with modules that do not require a lot of message event passing. For those fine-grained modules such as digital logic, number, and real number operation modules, decreasing the message passing speed will greatly hurt the performance. In this case, we won't be able to build data flow networks in NeatTools complex enough to meet the requirements of a general human interface tool. Other than the performance concern, every component in CORBA or COM/DCOM needs to dedicate some efforts and space on the interfacing, because the interfacing is where the object interoperability comes from. NeatTools has simplified broadcast models, as the interface is actually fixed and derived from the JModuleObj class. That means we have to dedicate identical interfacing by using IDL for each module, which is actually a redundant layer and could be eliminated. Thus, based on the space and speed constraints, I would suggest not using CORBA or COM/DCOM to develop NeatTools.

2.6     NeatTools and Modeling Languages

2.6.1     Colored Petri-Net

The following material regarding Petri-Net is extracted from the web page at the URL http://www.daimi.aau.dk/PetriNets/.

2.6.1.1     Introduction

Colored Petri-Net (CP-nets or CPN) is a graphical oriented language for design, specification, simulation and verification of systems. It is in particular well-suited for systems in which communication, synchronization and resource sharing are important. Typical examples of application areas are communication protocols, distributed systems, imbedded systems, automated production systems, work flow analysis and VLSI chips.

2.6.1.2     Why use CP-nets?

CP-nets are used for three different - but closely related - purposes. First of all, a CP-net model is a description of the modeled system, and it can be used as a specification (of a system to be built) or as a presentation (of a system to be explained to other people, or ourselves). By creating a model, we can investigate a new system before we construct it. This is an obvious advantage, in particular for systems where design errors may jeopardize security or would be very expensive to correct. Secondly, the behavior of a CPN model can be analyzed, either by means of simulation (which is equivalent to program execution and program debugging) or by means of more formal analysis methods (which are equivalent to program verification). Finally, it should be understood that the process of creating the description and performing the analysis usually gives the model designer a dramatically improved understanding of the modeled system -- and it is often the case that this is more important than the description and the analysis results themselves.

2.6.1.3     Analysis of CP-nets

CP-nets can be analyzed in three different ways:

The first analysis method is simulation. It is very similar to debugging and program execution. This means that we can execute a CP-net model (e.g., to get statistics about the behavior of the modeled system). It is possible to set breakpoints and to display the simulation results by means of different kinds of business graphics.

The second analysis method is occurrence graphs (also called state spaces or reachability graphs). The basic idea behind occurrence graphs is to construct a directed graph, which has a node for each reachable system state and an arc for each possible state change. Obviously, such a graph may become very large, even for small CP-nets. However, it can be constructed and analyzed totally automatically, and there exist techniques which makes it possible to work with condensed occurrence graphs without losing analytic power. These techniques build upon equivalence classes.

The third analysis method is place invariants. This method is very similar to the use of invariants in ordinary program verification. The user constructs a set of equations, which provides the satisfaction of all reachable system states. The equation is used to prove properties of the modeled system (e.g., absence of deadlock).

2.6.1.4     Design/CPN

Design/CPN is a tool package supporting the use of CP-nets. The Design/CPN tool is now distributed free of charge to all kinds of users (including commercial companies). Version 3.0 was released in May 1996. It has three integrated parts:

(1)      The CPN Editor supports construction, modification and syntax check of CPN models.

(2)      The CPN Simulator supports interactive and automatic simulation of CPN models.

(3)      The Occurrence Graph Tool supports construction and analysis of occurrence graphs for CPN models (also known as state spaces or reachability graphs/trees).

Design/CPN supports CPN models with complex data types (color sets) and complex data manipulations (arc expressions and guards) - both specified in the functional programming language Standard ML. The package also supports hierarchical CP-nets (i.e., net models that consist of a set of separate modules with well-defined interfaces).

2.6.1.5     Conclusion

From what I have observed, CP-nets are a good tool to simulate and verify software or hardware projects. But according to one of the Design/CPN introduction articles on the web, Design/CPN could do around 1,000 message broadcasts per second, which compared to NeatTools' 1,800,000 messages per second is around 1,800 times slower. As a simulation tool, it is OK to have a speed at this level. For real-time applications, Design/CPN may not be the proper choice. The key difference is in the broadcast model itself not implementation. CP-nets use asynchronous message broadcast between modules as the default. It could avoid the cyclic deadlock problem and be much like real world conditions. But in the mean time, it decreases the performance a lot. NeatTools uses synchronous message broadcasting and simplifies the broadcast into a direct function call. Also, it uses a simple mechanism to avoid the cyclic deadlock problem without a great deal of overhead.

In "An Introduction to the Practical Use of Colored Petri Nets," by Kurt Jensen, [19], he gives the following description:

CP-nets have an explicit description of both states and actions. This is in contrast to most system description languages which describe either the states or the actions -- but not both. Using CP-nets, the reader may easily change the point of focus from states to actions, or vice versa.

In my opinion, actions and states could all be expressed and manipulated by message events, because message events could stand for expressions of current states or actions. Actually, in NeatTools, I did implement the state as a module into NeatTools. The user could layout state machine visually and use it for complex state analysis. CP-net's decision to separate the modules into states and actions may have some advantage for theory analysis. But it could make the model itself become more complex and hard to understand for those users who have no formal theory analysis training.

2.6.2     UML (Unified Modeling Language)

The following material regarding UML is extracted from Rational's [21] web page at http://www.rational.com/ and OMG's [20] web page at http://www.omg.org/.

2.6.2.1     Introduction

The Unified Modeling Language (UML) is a language for specifying, visualizing, constructing, and documenting the artifacts of software systems, as well as for business modeling and other non-software systems. UML represents a collection of the best engineering practices that have proven successful in the modeling of large and complex systems.

2.6.2.2     UML definition

UML definition consists of the following documents:

(1)      UML Semantics: defines the rich semantics and expressive syntax of the Unified Modeling Language. UML is layered architecturally and organized by package. Within each package, the model elements are defined in terms of their abstract syntax (using UML class diagram notation), well-formedness rules (using text and Object Constraint Language expressions), and semantics (using precise text). Two appendices are included: UML Glossary and Standard Elements.

(2)      UML Notation Guide: defines notation and provides supporting examples. UML notation represents the graphical syntax for expressing the semantics described by the UML metamodel.

(3)      UML Extension for the Objectory Process for Software Engineering and UML Extension for Business Modeling: These UML extensions include process-specific and domain-specific extensions to UML, in terms of its extension mechanisms and process-specific diagram icons.

UML uses OCL, defined separately in the Object Constraint Language Specification document.

2.6.2.3     Development Project Artifacts

The choice of what models and diagrams one creates has a profound influence on how a problem is attacked and how a corresponding solution is shaped. Abstraction, the focus on relevant details while ignoring others, is a key to learning and communicating. Because of this :

(1)      Every complex system is best approached through a small set of nearly independent views of a model; No single view is sufficient.

(2)      Every model may be expressed at different levels of fidelity.

(3)      The best models are connected to reality.

In terms of the views of a model, UML defines the following graphical diagrams: use case diagram, class diagram, behavior diagrams, and implementation diagrams. These diagrams provide multiple perspectives of the system under analysis or development. The underlying model integrates these perspectives so that a self-consistent system can be analyzed and built.

2.6.2.4     Programming Languages

UML, a visual modeling language, is not intended to be a visual programming language, in the sense of having all the necessary visual and semantic support to replace programming languages. UML is, however, a language for visualizing, specifying, constructing, and documenting the artifacts of a software-intensive system, but it draws the line as you move toward code. Some things, like complex branches and joins, are better expressed in a textual programming language. UML does have a tight mapping to a family of OO languages, so that the user can get the best of both worlds.

2.6.2.5     Conclusion

Since UML is not intended to be a visual programming language, it is very clear that it has no conflict with the NeatTools application and its design concepts as a visual programming environment. On the contrary, we can use UML to help in specifying, constructing, and documenting the artifacts of the development of the NeatTools system or module design. The Rational Rose, a visual modeling tool that allows developers to define and communicate a software architecture, could be used to speed up the development of future module design for NeatTools. In this sense, UML could indeed be very useful to NeatTools development.

2.7     NeatTools and Other Visual Programming Tools

2.7.1     Microsoft Visual Series on C++, Basic, J++

The following material regarding Microsoft [8] Visual Series is extracted from Microsoft's web page at http://msdn.microsoft.com/visualj/, and http://msdn.microsoft.com/visualc/, and http://msdn.microsoft.com/vbasic/.

2.7.1.1     Introduction

Microsoft Visual Series uses the same Developer Studio environment that comes with Microsoft's Visual C++. Microsoft has done a good job incorporating its existing tools into VJ++ and Visual Basic. For example, to create dialog boxes, developers use the same dialog box editor found in VC++'s Developer Studio.

Windows developers will like the COM integration in VC++, VB, and VJ++. COM, Microsoft's component object model, is the core of ActiveX, and developers can incorporate existing ActiveX controls and COM-based technologies into their VC++, VB and VJ++ applications.

Developers can also build their ActiveX controls for use with Visual Basic, VC++, and Delphi. This is a fairly complex process, but the final release of VJ++ is expected to include an ActiveX Wizard that will make it easier to convert applets into ActiveX controls.

Developers could use application (or applet in VJ++) Wizard to create a small application. After they respond to the Wizard's few simple questions, it creates a commented source-code skeleton (and an HTML file in VJ++) to host the application (or applet). Thus, a user could start from there without the need to start from scratch.

2.7.1.2     Conclusion

Basically, I would consider Microsoft's Visual Series as a textual programming environment with some visual tools (like dialog box editor), ActiveX Wizard, and application Wizard to help a programmer integrate their applications with resources, ActiveX components, or to create skeleton source-code, etc. Conceptually, this is not relevant to the visual programming environment that NeatTools provides.

2.7.2     JavaBeans

The following material regarding JavaBeans is extracted from JavaSoft's [24] web page at http://www.javasoft.com/beans/index.html.

2.7.2.1     Introduction

The goal of JavaBeans is to define a software component model for Java, so that third party ISVs can create and ship Java components that can be composed together into applications by end users.

What is a Java Bean? A Java Bean is a reusable software component that can be manipulated visually in a builder tool. The builder tools may include a web page builder, visual application builders, GUI layout builders, or event server application builders. Or it may simply be a document editor that includes some beans as part of a compound document.

Some JavaBeans components may be simple GUI elements such as buttons and sliders. Other JavaBeans components may be sophisticated visual software components such as database viewers, or data feeds. Some JavaBeans components may have no GUI appearance of their own, but may still be composed together visually using an application builder.

Individual JavaBeans components will vary in the functionality they support, but the typical unifying features that distinguish a JavaBeans component are:

(1)      Support for "introspection" so that a builder tool can analyze how a bean works.

(2)      Support for "customization" so that when using an application builder a user can customize the appearance and behavior of a bean.

(3)      Support for "events" as a simple communication metaphor that can be used to connect up beans.

(4)      Support for "properties", both for customization and for programmatic use.

(5)      Support for persistence, so that a bean can be customized in an application builder and then have its customized state saved away and reloaded later.

While beans are primarily targeted at builder tools, they are also entirely usable by human programmers. All the key APIs such as events, properties, and persistence, have been designed to work well both for human programmers and for builder tools.

2.7.2.2     Linkage Model in JavaBeans

The linkage model in JavaBeans is naturally built on top of new Java 1.1 event broadcast mechanism and object class runtime information API. A bean programmer should follow proper design patterns to design interface method and property signature. So, the bean builder could study a bean's supported properties, events, and public methods through a low-level reflection mechanism or via BeanInfo class. In order to broadcast events between beans, bean builder first obtains the source bean, destination method, and event type information through its user interface. Later, it has to create a linkage class dynamically. The linkage class will register itself as an event listener to the source bean and also acts as a bridge between the event source and destination method. So, when the source bean broadcasts an event, the destination method will be invoked through the linkage class. The linkage building process does not happen completely in run-time, because the dynamically created linkage class and other generated source files needs to be compiled before the application could be executed.

2.7.2.3     Conclusion

The most noticeable differences between NeatTools' module model and JavaBeans' component model are introspection process and linkage creation.

JavaBeans use a low-level reflection mechanism or BeanInfo class to study the methods supported by a target bean. In Java, the low-level reflection mechanism is already built in the system API. For example, the object.getClass method returns a object, which describes the current object instance. Through the object, we could walk the class hierarchy or determine the signature of a particular method. In C++, this kind of mechanism is almost impossible without an extensive implementation to embed the detailed run time object information automatically. In NeatTools, the introspection process is not needed, because methods signature are fixed and inherited directly or indirectly from the abstract module base class. By deriving class from a generic data type class, methods with fixed signature could be used to broadcast events with different data types. Hence, the same functionality could be achieved.

In NeatTools, the linkage creation process is very simple and efficient compared to JavaBeans. The linkage data structure is fixed and it records the source module, source port number, destination module, and destination port number. No linkage class creation and compilation is involved during the linkage creation process. When a user requests to create a connection by mouse drag and drop, the NeatTools desktop just initializes a linkage data, fills in the information, and adds it into the linkage set. Unlike NeatTools, bean builder usually does not have a visual linkage layout representation. Instead, the linkage information between beans is accessed through the pop up menu or separate data entry form.

In general, JavaBeans' linkage and event broadcast model is very flexible and scalable. It takes advantage of some special features inside Java API, which is missing in C++, to assist the introspection process. Instead of using run time object class information, Object oriented programming technique was used to implement the module abstraction and event broadcast, which is simpler and more efficient. Since we decide to implement NeatTools in C++ and JavaBeans' linkage model is not suitable for language other than Java, NeatTools' module abstraction and event broadcast architecture is the reasonable implementation choice.

2.7.3     LabView

The following material regarding the LabView software package is extracted from National Instruments' [23] home page at http://www.natinst.com/.

2.7.3.1     Introduction

LabView is a graphical programming development environment based on the G programming language for data acquisition and control, data analysis, and data presentation. LabView gives a user the flexibility of a powerful programming language without the associated difficulty and complexity because its graphical programming methodology is inherently intuitive to scientists and engineers.

With LabView, a user builds VIs (virtual instruments) instead of writing programs. Users can create front panel user interfaces, giving them interactive control of the software system. To specify the functionality, a user assembles block diagrams - a design notation for engineers and scientists.

2.7.3.2     Data flow Programming

LabView implements a patented data flow programming model, called G, that frees the user from the linear architecture of text-based languages. Because the execution order in LabView is determined by the flow of data between blocks, and not by sequential lines of text, users can create diagrams that have simultaneous operations. LabView is a multitasking and multithreaded system, running multiple execution threads and multiple VIs.

2.7.3.3     Graphical Compiler

In many applications, execution speed is critical. LabView provides the compiler that generates optimized code with execution speeds comparable to compiled C programs. With the built-in Profiler, a user can analyze and optimize time-critical sections of code.

2.7.3.4     Multithreading

In the new version of LabView, the multithreading is built into VI, or LabView programs, thus it is not necessary for users to learn new programming techniques. In fact, the user can benefit from multithreading without even knowing what it is. However, for expert users who want to have specific control over threads, such as changing thread priorities, that flexibility is available in a straightforward dialog box option.

2.7.3.5     Conclusion

NeatTools and LabView are both visual programming environments. NeatTools is based on the abstract module model and LabView is based on the G programming language. The abstract module model tries to generalize and simplify the module and event broadcast model from a pure visual programming perspective, while the G programming language tries to have the graphical representation of traditional textual programming. For example, you can see the "while loop" and "for loop" structures in G programming language. In NeatTools, we do not define "while loop" or "for loop" structure in module specification. All modules are logically equal and they all derive from the same module base class. In one word, NeatTools users could use the Timer or ClockDivider modules to provide the same functionality. Timer and ClockDivider are just regular modules. Basically, a NeatTools user could design and use anything he likes to accomplish the desired tasks.

The graphical compiler in LabView provides optimized code with execution speeds comparable to compiled C programs. This indicates that the simulated version in LabView is slower and much bigger than the compiled version. In NeatTools, as I discussed in previous sections regarding NeatTools' performance, its data flow network is already comparable to compiled C/C++ programs. Also, NeatTools executables only include the core modules, thus the user could just add functionality by including the external module DLLs needed to meet the space constraints. Hence, the compiler is actually not needed in NeatTools.

Recently, the new LabView version started to support multithreading. The thread itself is associated directly in the VI. A user can change the thread's priority through a dialog box. I do not think this kind of add-on threading could provide a user all the power and flexibility of threading. In NeatTools, every module could start their own thread (or multi-threads) depending on the tasks it needs to perform. By using connections, several modules could share threads together, or modules could use internal threads for calculation intensive tasks, etc.

LabView has a very strong user group and history going back over ten years. NeatTools originated about 2 years ago. NeatTools has some technology advantages over LabView owing to its fairly new structure and design. I believe, with proper promotion, NeatTools could become popular in the educational market as well.

2.7.4     AVS/Express

The following information regarding AVS/Express is extracted from the home page of Advanced Visual System Inc. [22] at http://www.avs.com/.

2.7.4.1     Introduction

AVS/Express is a tool for developing data visualization applications. It enables users to explore data sets of any dimension or complexity using visualization capabilities.

AVS/Express offers the same benefits as other advanced client/server development tools. It provides a multi-platform, multi-OS development environment that allows broad-based application building. It enables developers to spend their time adding value to applications instead of trying to solve non-productive, costly system-level issues.

2.7.4.2     The Graphics Display Kit

The AVS/Express Graphics Display Kit is a set of objects that contain the data structure and function to develop data-intensive, interactive graphics applications. It provides the following graphics-component technology essential for rendering and manipulating text, 2D images, and 2D and 3D geometric objects. These components can be reconfigured, customized and easily replaced to provide custom views for end-user applications.

2.7.4.3     The Data Visualization Kit

The Data Visualization Kit contains objects, data structures and libraries needed to visualize and analyze data sets. The data model within this kit defines how data is represented, how functions access it, and how it is communicated between functions and processes. The AVS/Express data model could handle the data frequently found in data and graphics intensive applications, including image, volume, finite element, scattered and geometric data. The visualization components in this kit contain computation methods that are key to turning abstract data sets into graphics.

2.7.4.4     The Database Kit

The Database Kit provides interfaces to the most popular SQL-based relational database management systems (RDBMS), including Oracle [15], Sybase [35] and Informix. In addition, the data kit contains an Open Database Connectivity (ODBC) interface. A library of configurable visual objects provides connections to one or more relational databases, and enables display of database tables and assembly of SQL expressions.

2.7.4.5     Visual Programming

In AVS/Express, users construct applications as a connected, hierarchical network of objects, create and modify data structures with "drag-and-drop" ease, manipulate widgets directly to create GUI layouts, set up GUI callbacks through visual connections between widgets and application methods, examine application state and data with integrated debugging tools, and integrate external functions with encapsulated C, C++ and FORTRAN routines.

2.7.4.6     Conclusion

NeatTools is meant to be a generic fine-grained visual programming tool for developing human computer interfaces. NeatTools' focus is on how we could increase the information flow throughput between humans and computers. Thus, the key issues are on how we provide more information paths from humans to computers through sensors, devices, and broaden all kinds of feedback from computers to humans through graphics, audio, video, touchware, etc. It is possible that we could write all kinds of AVS/Express modules to provide the functionality specific to the NeatTools project. But, we never could change AVS/Express system itself to meet some special constraints and solve the problems specific to the NeatTools project. Due to different design concepts and implementation architectures, AVS kernel provided the centralized control over the inter-modular communication, and hence serialized parallel channels. Also, AVS uses IPC data-flow protocol to implement the channel event broadcast and data exchange, which limits AVS as a coarse grain visual programming tool. Thus, AVS could be very convenient for coarse grain integration such as manipulate and control the application flow of modular processes distributed in different hosts. When it comes to hundreds of modules connected by thousands of connections with high frequency event broadcast, what we need is an architecture that not only provides a high performance inter-modular communication, but also unleashes the power of well designed fine grained modules in the hands of a user. NeatTools fits into this kind of scenario in human computer interface very well and it provides flexible solutions as a general fine-grained visual-programming tool.

3.     Summary of NeatTools Implementation

3.1     Design Goal of NeatTools

The design goal of NeatTools is to make it a simple, object-oriented, network-ready, robust, secure, architecture neutral, portable, high-performance, multi-threaded, and dynamic platform for human computer interface.

3.1.1     Simple

3.1.1.1     Simple Usage for the Data-flow Network Designer

The goal of building a system that could be visually programmed without a lot of formal training in computer science was achieved. At the SigGraph Conference 98', a very large booth in the SigKids area allowed children (around 10-12 years old) to freely use a full version of NeatTools interactively as an experimental tool. Most children could pick up the concepts of NeatTools in a very short time and started exploring on their own. One of them was definitely the best of the best. He picked up all the major ideas in a half-hour and started to design his own data-flow networks ranging from a "metal detector" to "mouse position finder." This proves that the NeatTools visual programming environment design is naturally simple.

I added several concepts to NeatTools, which are based on purely visual programming aspect instead of the traditional textual programming. For example, most visual programming languages like AVS/Express and LabView, have distinct modes for data-flow network editing and execution, which are quite naturally accepted in traditional textual programming. Because every textual program source code needs to be edited and compiled before it can execute. Also, from the design point of view, making editing and execution mode exclusive will make the system design task easier. The mixture between editing and execution mode will cause a lot of technical difficulties especially in a multi-threaded system. NeatTools provides non-stop execution. So users can layout, execute, and debug data-flow networks at the same time without being interrupted.

3.1.1.2     Simple Programming Paradigm for the Module Designer

In module design, a programmer will first identify the module object, which will be used by the data-flow network design as the basic building block. Usually, this portion is most difficult because it will require broad knowledge of object-oriented programming and visual programming environment design. After the module object has been identified, the programmer could proceed to create a class which derived from the module base class or other module (when needed to inherit the common behavior). In terms of the process, creating a NeatTools module is very simple and straightforward.

3.1.2     Object-oriented

Object-oriented design is very powerful because it facilitates clean definition of interfaces and makes it possible to provide reusable software ICs. By software ICs, I mean the design hiding the implementation and information and only providing the interface as the communication media to the outside world. So the designer could focus on the design and implementation of software ICs without additional concerns.

In NeatTools, each module was designed as a special object unit. The object unit defines the module's visual representation, connection channel, connectivity of each channel, attribute, etc. All the interactions between modules are handled by the module's base class and NeatTools system. So the designer can focus on each module as an independent unit and may improve the algorithms or debug inside the module.

3.1.3     Network-ready

NeatTools integrates an extensive library of TCP/IP protocol routines into the visual programming building blocks. For examples the SocketObj and ServerSocketObj modules can be dropped into the desktop and create a network connection by just making connections which is much easier than in a traditional textual programming environment. In the future, to complete the network functionality, we will add more modules that can handle HTTP, FTP, and other network related protocols.

3.1.4     Robust

Most dynamic languages like LISP, TCL and Smalltalk are often used for prototyping. One of the reasons for their success at this is that they are very robust: we don't have to worry about freeing or corrupting memory. In Java, the programmer can be relatively fearless about dealing with memory because they don't have to worry about it getting corrupted. In C/C++, the pointer and memory management are the most problematic issues. So how could we make the NeatTools become robust and still use C/C++? I believe proper usage of object oriented programming is the key issue to achieve this goal. By hiding the pointer and memory management inside the data members, constructor, destructor, and assignment operator methods; the interface of object class could be free of pointer and eliminate the need of complex memory management -- it becomes distributed into each object class implementation.

 

3.1.5     Secure

NeatTools is designed to be used in network/distributed environments. By using the Socket/ServerSocket modules, NeatTools running on different computers could collaborate with each other through data exchange and events. Recently, a lot of emphasis has been placed on security. To enable a secure visual-programming environment, the addition of authentication modules that are based on public-key encryption was planned, users could use the authentication module to encrypt or decrypt events and close the door on most tampering activities.

3.1.6     Architecture Neutral

In the present computer market, application writers have to produce versions of their applications that are compatible with the IBM PC [36], Apple Macintosh [37], and other high-end workstations. With the PC market (through Windows/NT) diversifying into many CPU architectures, Apple moving off the 680x0 toward the PowerPC [37], and all the different brand named workstations based on different CPU architectures, production of software that runs on all platforms becomes nearly impossible.

In Java, the same version of the application runs on all platforms. But the Java bytecode relies on the Java runtime that allows the Java virtual machine to run the byte code. As demands come from all the different industries, Java runtime is constantly revised and becomes bigger and bigger. Because of that the browsers that carry Java virtual machine becomes huge, and they waste a lot of space when different Java runtime instances are needed by different browsers or Java development tool kits.

Some existing solutions, like the Win/U from Bristol Technology [38], provide biased implementations that favor a particular platform. For example, a lot of Microsoft Windows applications were written based on the Microsoft Foundation Class (MFC) [8], which is built on top of Win32 API. Win/U provides a layer that simulates the Win32 layer on the UNIX platform. So applications based on MFC could be compiled and run on the UNIX platform. This solution is based on the fact that lots of existing applications are already written for the Microsoft Windows platform. But it introduced unnecessary layer into the UNIX platform and decreases the application's performance.

NeatTools is built upon a Java Like Cross Platform API. In this API, it defines the basic language, networking, input/output, utility, and abstract window classes. It hides all platform dependent implementation details in the classes, provides a unified class interface to the applications that are based on this API and does not favor any of the platforms. By using this architecture neutral API, we only have one very thin layer between the application and the operating system it is running on. It guarantees that the application becomes cross-platform and will also have high performance.

3.1.7     Portable

Being architecture neutral is a big chunk of being portable, but there's more to it than that. All the platform dependent details are hiding in the object implementation. What the object interface provides is unified to all platforms. This is the classic usage of the object-oriented feature to build a clean portability boundary between the C/C++ POSIX subset, operating system, and applications.

3.1.8     High-performance

The architecture neutral, portable, Java like cross-platform API did provide the thin layer that ensured the high-performance of the NeatTools that built upon it. NeatTools' visual programming environment design is based on the module abstract model which simplified the object interaction into input and output actions. To provide the highest performance, NeatTools uses distributed messages broadcasts and direct function calls to implement the event broadcasts between modules. Unlike AVS, which uses the AVS kernel to provide centralized control over inter-modular communication and uses IPC data-flow protocol's RPC call to implement the channel event broadcast and data exchange, NeatTools has much superior performance over AVS on event broadcasting. According to the benchmark, NeatTools is capable of broadcasting 1,800,000 events in one second, which is not even possible for an AVS application.

3.1.9     Multi-threaded

There are many things going on at the same time in the world around us. Multithreading is a way of building applications with multiple light-weighted processes. These light-weighted processes share the same data segment and execute concurrently. Unfortunately, writing programs that deal with many things happening at once can be much more difficult than writing in the conventional single-threaded C and C++ style. By using the Java like cross platform API, a sophisticated set of utility classes that provide the thread operations and synchronization primitives is provided. In Microsoft windows, the underlying implementation is based on Win32 Thread API. In UNIX, the underlying implementation is based on the MIT POSIX Thread software package.

In NeatTools, the interactive responsiveness and real-time behavior is very strong. All threads inside modules will not be interrupted or suspended by user operations like layout modules or making connections between modules. Everything is working concurrently and smoothly when the user is trying to group, resize, or move a set of modules. The multi-threaded feature in NeatTools did increase the productivity, functionality and usability.

3.1.10     Dynamic

To adapt to an evolving environment, NeatTools was designed to be extensible. External modules could be added into the system at runtime, which could extend the system, as new modules are available. In NeatTools, each module is a distinct object class, which could be put in the system statically as a build-in module or could be linked into the Dynamic Linkage Library (DLL) and, later on, loaded by the NeatTools system as an external module. The Polymorphicism feature in Object Oriented Programming makes it possible for NeatTools to be extensible. The virtual function provides a way that enables a proper method implementation pointed to by a base class's virtual function table. So when a module was been dynamically linked, we could create an instance of the module and later on express its behavior through the common virtual functions inside the module base class.

3.2     NeatTools' Visual Programming Features

3.2.1     Introduction

Developing an application in NeatTools is very easy. Here is an example that shows how you could use mouse clicks to light up an LED. First, launch NeatTools, and set the desktop to "Edit" mode by pressing on the "ED" button in the left column. After opening the "Display" toolbox by pressing on the "DS" button, you are ready to start programming.

Now, move your mouse over to the button module and drag it to desktop. Then, move the mouse over the LED module and drag it to desktop also. Lastly, move the mouse over the output area of the button and drag it to the input area of the LED module. You will see a new connection built between the button and LED modules. Your first NeatTools program is ready and running. If you click on the button module, you will see the LED module light up. (Like in the following picture):

Figure 8: NeatTools Introduction

You can save your program by press the "SF" button and assign it to a file with an "NTL" extension. You always could load your program by pressing the "OF" button and selecting your NTL file to bring it back.

3.2.2     Multi-Thread Features

A NeatTools module can initiate its own threads depending on its design. A NeatTools user could just use it without notice. For example, you could put Time and Date Display modules on the desktop, and connect them. Now, you will see the Date module is showing the current time and updates every second. Then, you put the Timer, ClockDivider, and LEDs to the desktop, and connect them. You see the LEDs light up and cycle through each other. Right now, two threads are running at the same time inside the desktop area. By sharing the events, modules may share the same thread. For example, the four LEDs and ClockDivider share the same thread inside the Timer module.

Figure 9: Multi-thread features

3.2.3     Keyboard and Mouse Event Simulator/Filter

NeatTools could receive Keyboard or Mouse events through Keyboard or Mouse modules' outputs. In the following example, we use the keyboard "1" to control the LED and use 1DViewer to display the mouse X position.

Figure 10: Keyboard/mouse event simulator/filter (1)

NeatTools could generate Keyboard or Mouse events through the Keyboard or Mouse modules' inputs. The computer will "think" that someone is typing the keyboard or moving the mouse, but actually those events are generated by NeatTools. The following example uses mouse X position and 1DViewer to fire the keyboard module when mouse X position is in the right area. If a user launches an editor like Notepad or Word and puts it in the foreground, you will see a "1" character appear when you move the mouse from left to right.

Figure 11: Keyboard/mouse event simulator/filter (2)

This function is very important and useful. For example, if we want to help a disabled person to use a sensor to surf the Web, we can use NeatTools to pick up the signal and process it into the mouse or keyboard simulated events. Or, if we want to make NeatTools work with an existing application and can not change the application itself, we can program NeatTools to generate keyboard or mouse events to use and control the application. For fast prototyping purposes, this function is extremely useful because we always could connect and control an external application in no time.

3.2.4     Networking and TCP/IP

Transferring information to remote computers is very easy, after you layout the socket server and socket client, set the IP address, connection port, and enable both of them; you already have built an invisible duplex connection between these two modules. The socket and server socket pair can be put into different NeatTools processes or in a remote computer as long as the IP address is correct. With these Networking modules, a user can transmit any NeatTools data type to any computer running NeatTools. The data could be an integer, floating point, bytes, wave, or video data. Laying out an Internet phone using NeatTools is very easy also.

Figure 12: Networking and TCP/IP

3.2.5     Container Nest Structure (Complex Module)

NeatTools could do visual modular design by hiding data flow networks inside a container module and becoming a complex module. Inside the container module, you can put in complex modules of other data flow networks. So a user could design an application using several layers. Each layer will handle different levels of complexity. This feature is rarely seen in most visual programming languages.

Figure 13: Complex module

3.2.6     Transfer Focus among Text Fields

How to provide a full functional graphical user interface visually is always an interesting topic. For example, a text field module is needed, which could let a user type in characters and compose them into a character string. Usually, most applications will have several text fields for a user to type in. How do we provide a mechanism, which enables a user to transfer focus among text fields? Using mouse clicks to set the current focus would be an obvious way to do it. But for the user's convenience, some short cuts for the user to toggle or cycle through all the text fields or GUI components need to be provided. To implement the short cut feature, I added an input port called editFocus, which could trigger and set to the current focus by other modules. I also added an output port called nextFocus, which will send out a true event when it is in focus and the user presses the TAB key. By connecting the nextFocus output port into the next text field's editFocus input port, we could build a form with several text fields and a user could use the TAB key to cycle through the fields.

Figure 14: Transfer focus among text fields

3.2.7     Polymorphic Data Type

NeatTools is a strongly typed visual programming language. Usually, it is desirable to connect input and output ends only when they are of the same data type. The NeatTools system guards that very well, but sometimes, it is not so convenient to implement all the different modules for each different data type. The ControlObj module is one of the modules that needs the polymorphic data type. ControlObj module works as a switch controlled by enabling input signals on the top, which is very useful for all kinds of data. With polymorphic data types, the ControlObj input/output pairs could connect to any data type. For example, after connecting the data input port to an integer port, the input/output pair adapts itself into the integer data type. Then the corresponding output port could accept integer data. With the polymorphic data type, we make a balance between strong typed and weak typed checking and make NeatTools more convenient and efficient. In the following example, I use ControlObj to control integer, date, and real number at the same time.

Figure 15: Polymorphic data type

3.2.8     Multimedia Features

NeatTools has added most of the multimedia features into modules including Mixer, Wave input/output devices and MIDI input/output devices. Now, a signal that comes in from any device or remote computer can control the CD volume, play a MIDI song, hear sound wave, etc. The information perception could be anything we ever think of; we could "hear" the image, or "see" the sound. We plan to add more modules into NeatTools to handle graphics visualization and data processes. In the information age what we need is not just information, we need all kinds of different ways to observe information and abstract the patterns and features we want.

The following is an example in which NeatTools is used to visualize sound wave intensity.

Figure 16: Multimedia features

3.2.9     Multimedia Database

Most people consider a database as belonging to a textual programming API layer, but I personally think providing a visual database is a very interesting concept. Different databases could use connections to build the relations between databases and become a visual relational database network. The relations between databases could be indexing, key word match, or condition match. The indexing relation is quite straightforward, users just connect an integer field value to the index input of another database. Whenever the integer field value changes, it will locate the connected database into the corresponding record. The keyword match and condition match relations work likewise.

In the following example, the Session database file contains several fields including patient's SSN and therapist's Reference Number. Session data uses the SSN field to build a relation with the Patient database by connecting from its SSN output to the Patient database's search of SSN input. Session data uses Therapist_Ref field to build a relation with the Therapist database by connecting from its Therapist_Ref output to Therapist database's search on Ref_No input. Therefore, when we navigate the Session database by changing its index, the Patient and Therapist_Ref database will move to the corresponding record.

Figure 17: Multimedia database

3.2.10     External Module and Dynamic Link Library

In NeatTools, modules can be built-in or external. The external module will be a dynamic link library. NeatTools could load the external module in run time. The external module could be compiled separately, so NeatTools becomes scalable. Users could expand NeatTools' functionality by adding their own modules. Also, some device manufacturers provide their own API in DLL. If we make an external module DLL that includes the manufacturer's API, we could reduce the NeatTools system's dependency on the manufacturer's DLL and keep the NeatTools system core compact and small.

The following picture shows the dialog box that NeatTools uses to load the external modules.

Figure 18: External module

3.2.11     State Machine

In many control-based applications, the state diagram is the dominant aspect of the model that can capture the dynamic behavior. The major concepts of dynamic modeling are events, which are the stimuli, states and configurations of object behaviors. In NeatTools, events are conducted by connections between modules, and states are represented by a special State module which can hold a single named state and can stimulate the connected state modules when conditions are matched. A State module can have several outgoing connections to other state modules. Each connection can only connect to one state module and can be stimulated by one condition pulse input event.

The following is a very simple example of a state machine. There are 3 states, A, B, and C. Each state could be triggered by a condition and stimulate the next connected state module. A state module could be setup to trigger by several conditions and connect to several other state modules. I will put a detailed example in next section.

Figure 19: State machine

3.3     NeatTools Architecture

The NeatTools architecture defines the complete structure for implementing NeatTools' cross-platform, extensive module unit, usability, and functionality features by using UML (Unified Modeling Language).

3.3.1     Three-Layer Architecture

The NeatTools' implementation model is defined as a three-layer architecture. This architecture is necessary for cross-platform design. There are several advantages associated with this approach:

(1)      It separates the application design from platform dependent details by providing abstract windows and a system resource layer for building the application upon the abstract layer.

(2)      Based upon a well defined interface in the class, a system programmer can refine the design and algorithms later without needing to change the dependent layer.

(3)      Current commercial window abstract layers, like Microsoft MFC, Borland OWL, and OSF MOTIF tend to be fat in design and fixed on one specific platform.

The conceptual framework for NeatTools is based on architecture with three layers:

(1)      Operating system and programming language layer

(2)      Java like cross-platform API layer

(3)      NeatTools application layer

Functions of the layers are summarized in the following table:

Layer

Description

Operating system and programming language

Services and resources provided by operating systems and programming language runtime library.

Java like cross-platform API

Set of abstract classes that provide the windows and basic services for application construction needs.

NeatTools application

Provides the actual implementation of usability and functionality, which meet the general requirement as a real-time visual programming tool.

Table 6: Summary of layers

The operating system and programming language layer provides all the basic system services and resources. This layer includes Win32 API, X11 API, and C++ run time library. The Win32 API and X11 API provide the generally required functionality of windowing, graphics context, objects and memory management in a platform dependent fashion. The C++ run time usually comes with the actual compiler implementations. There are even some standards proposed for the C++ language itself.  However, there are still some significant differences between companies and platforms.  Hence, it is desirable to have an abstract layer, which provides standardized interfacing between the application layer and the operating system layer.

The Java-like cross platform API layer is the abstract layer that works between the operating system and the application. It releases the application's dependency on the operating system and provides a high-level utility object support. Usually, the API is a collection of procedures. It is usually very difficult for programmers to  find out the relevant procedures in an API; know its capability and usage; know how it interacts with other procedures; not to mention putting them together in order to accomplish a required task. In this sense, a cross-platform API layer is necessary and desirable.

Now comes the second question: why Java-like. "Java-like" means it defines the class and interface according to a Java API specification. NeatTools needs to be fast and efficient to handle real-time and computationally intensive tasks. By implementing a Java-like API in C++, we gain Java’s ability to run on different platforms as well as the high performance of C++.  The application layer is where we build the actual NeatTools implementation in cross platform fashion. Different applications will eventually need some special classes which could perform the usability and functionality needed for the particular applications. So the presence of this layer is natural.

3.3.2     Package Structure

The NeatTools implementation is moderately complex. It is composed of approximately 100 classes in a Java-like cross platform API layer and approximately 120 classes in a NeatTools application layer. The complexity of the classes is managed by organizing them into logical packages. The NeatTools implementation is decomposed into the top-level packages shown in Figure 1:

Figure 20: Top-level Packages

3.3.3     OS and C++ Runtime Package

The Operating system and C++ runtime package is further decomposed as shown in Figure 2:

Figure 21: OS and C++ Runtime Layer

3.3.4     Java-like API package

The Java-like cross platform package is further decomposed into NET, AWT, UTIL, IO, and LANG packages as shown in Figure 3:

Figure 22: Java like Cross-Platform API Layer

The exception classes structure inside the Java-like cross platform packages is illustrated as shown in Figure 4:

Figure 23: Exception Class Diagram in Java Like API Layer

4.     Experiments

4.1     Case Descriptions

The following two cases were taken from the pulsar web pages. Eyal’s page is at http://www.pulsar.org/eyal/index.html and Brook’s page is at http://www.pulsar.org/brooke/index.html.

4.1.1     Eyal Sherman

When he was 5, Eyal Sherman suffered from a stroke. After surgery, Eyal was a quadriplegic, with severely limited movement. Communication was restricted to yes and no questions or lip-reading. It had been 10 years since this tragedy, and life had settled into a kind of norm for the Shermans. Eyal attends public schools, where he is enrolled in New York State Regents classes, a program for advanced students.

4.1.2     Brooke Kendrick

Brooke is a delightful 7 years old who is diagnosed with spastic quadriplegia due to cerebral palsy. She functions at a pedal gross motor level. She uses a power wheelchair with a joystick control, which she is presently learning to use. Brooke is non-verbal but does make her desires known by smiling, laughing, crying and saying yes and no. She is a social little girl who appears well aware of what is going on around her. She continues to demonstrate a strong startle reflex when loud or unexpected noise is heard. Brooke attends an integrated First Grade program at Lakeland Elementary School. This is a full day program, which integrates her educational and therapeutic activities. She has a one-on-one teaching assistant.

4.2     Approach

The NeatTools project team has been working with Eyal Sherman closely since 1996. The approach was divided into hardware and software portions. The hardware team developed the EMG (electromyograph) detector known as TNG and did experiments with all kinds of headsets and mounting systems for Eyal. They also tried a lot of sensor technologies including photocells, displacement potentiometers, Hall Effect transducers (magnetic sensors), pressure transducers, bend sensors, etc. Jo Johanson was the one who mainly developed the NeatDOS [28]. Later on, he briefly experimented on a Windows and Java version. TNG devices pick up the signal from the sensors on the headset and transmit them into the computer. The Neat software keeps receiving the signals, processing them, and later on maps the signals onto actions of external devices.

4.3     Limitations of Early Software

The old design, especially the DOS version, is a great tool to receive signals from external devices, change the configuration dynamically by making connections, and then do logic operations on the buttons. The major draw back was that modules have a somewhat limited layout. The format is fixed and could not provide the full flexibility for user to layout. For example, a user could add new buttons. After button creation, a user would see a new button appear at the end of the button area. A user could add connections to a button, but could not move the button or resize the button itself. Also DOS does not support multi-process, multi-thread, TCP/IP, and multi-media, as there are many limitations inherent to the DOS operating system.

The Java version improved the layout mechanism. Users could create new modules and put them anywhere on the desktop. This version uses a native method to implement the COM port related function and runs under Java enabled browsers. But, as the JDK changed, the new browsers prevented Native methods from low level COM port access for security reasons. Without COM port access, the Neat software could not read the signals from the TNG device and so would sit in the dark. Other than the COM access problem, there is no simple way of communication with other existing applications. For example, to enable the user to employ a TNG signal to control a Java Tetris game, the software team member has to find out the source code, download it and rewrite the code. Thus, Neat software could create and control the game directly, but when it comes to commercial software applications like Netscape and Word, there is no way we could acquire the source code and try to modify a huge application and combine it into Neat software. This approach is not feasible at all.

4.4     Ideas and improvements

At this point, the group tried to find a Java programmer to solve the problems. On one evening, when Eyal had a session with the NeatTools project team, I came by to visit them and tried to understand the nature of the project and the problems they were facing.

4.4.1     The mouse driver prototype

The first thing I noticed is that to try to modify the source code of applications in order to control them is not realistic. There are hundreds and thousands of applications out there. How could we modify them all? Also, most commercial applications will not release their source code. Then how do we provide a way for disabled persons like Eyal to use them? Secondly, Java is changing so fast, and its future is still not certain yet. Right now, the new browsers prohibit us from COM port access in the native method, and JDK itself does not provide the functionality to access the COM ports, (and will not in the future, because it is meant to be a cross platform language). Why should we limit ourselves by using Java? I happened to have the knowledge of how to simulate mouse events and how to install a mouse event filter in the Windows environment, so I decided to give it a try.

Three days later, I came out with a very small C windows program (executable around 60k). It has some simple controls and property settings on its front panel. After it connects to the COM port, the signal from TNG will come in and display in its client area. By changing the signal, it could generate mouse events, change mouse position, and simulate mouse click. When we put this small program into the background and brought other applications to the foreground, we were able to use the signal from a TNG to control any application without having to rewrite it!

This prototype mouse program solved some of the major problems, but it was still not good enough to generally adapt to changing situations. For example, I hard coded the mapping between a user gesture signal and the mouse events. But, if a patient could not do some of the predefined gesture actions and needed to change it into other gesture actions, then I would have to expand the program to make the random gesture actions/mouse events mapping possible. Also, TNG is just one of the input devices, there are many different devices out there, thus how could we provide a generic way for a user to configure all of them easily?

4.4.2     Next Generation Neat Software - NeatTools

To make it adapt to changing situations, the data flow network with a visual programming environment is the way to go. This time, I wanted to make it general and powerful. I devote the next section to discussing the key issues in building a better Neat software.

4.5     Key Issues in Building Better Neat Software

4.5.1     Java Like Cross Platform API in C++

The Java Like Cross Platform API in C++ is actually a very thin layer, which is provided as an interface to the operating system, C++ runtime library, and Windows system. This API hides the platform dependent implementation and unhooks the application's dependency from the operating system and Windows system.

Other than following the JDK standard, the challenge was actually in how to map the Windows handle, related operations and Window messages into the object that represent the window object. It has to provide an efficient way to associate the window events into object's methods and a unified way to map the same object into window objects of different windows system like Microsoft Win95/NT and UNIX X Windows.

Usually, programmers of Microsoft Windows will use the MFC to ease the complex process of registering Windows class, setup parameters, create windows, and finally display it and start to process window events. By using MFC on UNIX, a layer must be provided to simulate Win32 API on top of X Lib (and MOTIF sometimes). The Win/U product by Bristol Technology was one of the products that provided the cross platform solutions on UNIX. But this solution has a bias on UNIX platforms, because an application will have two layers between itself and X Lib, which needless to say will decrease the application's performance on UNIX platforms. So, to be fair to both major Windows operating systems, Java-like Cross Platform API in C++ is the key concept for NeatTools to become a cross platform and high performance application.

4.5.2     Module Coordinate System

Most visual programming environments use integers to represent the current position and dimension. They can not scale the desktop, or scale only in limited settings. Eventually, when data flow networks become complex, a user could use up all the space and have to overlap the modules and connections. I proposed using a floating point instead. This change enabled the desktop to have arbitrary scale. A user could have a layers design, where each layer has different visibility and could handle the complexity locally. A module will know its current screen dimension and position in integer pixel units only when its repaint or update method are invoked. NeatTools not only supports a scaling system that could scale a design to arbitrary scale, but also provides rulers and scroll bars to help users locate or reposition a design without the limitations of screen resolution.

4.5.3     Module Event Broadcast Model

As I discussed in the previous section about NeatTools' implementing concept, the most important key to a high performance fine grain visual-programming environment is how the event broadcast performance is increased.

In NeatTools, the module abstraction ensures that there is almost no dependency of one module on another, except for the data objects that flow in and out of a module, software modularity and reusability is enforced and highly preserved in the system. The same idea was presented in most visual programming environments, but different implementation architectures could lead to dramatic performance differences based on the same idea. For example, in AVS, the AVS kernel provides centralized control over the inter-modular communication. Restricted by the AVS kernel and its IPC data-flow protocol, any communication between two parallel modules has to go through the sequential host-node channel and the module-module pipeline on the control processor. In this sense, the current AVS kernel serialized parallel channels. This could become a performance bottleneck. Not to mention the IPC data-flow protocol's RPC call is much slower than a direct function call. AVS kernel design has its basis in the distributed nature of AVS modules. But this design limited AVS as a coarse grain visual programming tool. All the fine-grained operations and functionality have to be implemented inside the module and can not be exposed and utilized by the user directly through the visual programming environment.

In contrast to AVS kernel's centralized broadcast control model, NeatTools not only has a centralized storage for all logical connections but also every NeatTools module has a local reference to its logical connections (including input and output connections). When broadcasting events, NeatTools modules can make direct access to logical connection information and invoke the destination module's engine method directly. Different modules could issue parallel events under different threads. This is the distributed broadcast model in NeatTools and proves it could meet the high performance requirements of a fine grain visual-programming environment.

4.5.4     Increase Usability

In most visual programming systems including LabView and AVS/Express, when in edit mode, a user could put in new modules, make connections, copy, or delete modules, etc. When editing is finished, the user will have to press a button or select an option to tell the system to enter the execution mode, observe the execution and, later on, stop it and resume edit mode for the next development cycle. When the program is in execution mode, a user can not change the layout, connections, or modules. From a programmer's point of view, it is always technically easier to separate the operations into edit and execution mode exclusively. Thus, the implementation could concentrate on a single scenario without having to worry about the interactions between different operation modes. Yet, from the user's point of view, somehow this kind of distinction increases the time of development. It is quite often that the user could make some changes in edit mode and start the execution mode. Then he observes the problems or bugs and could not modify the network right away. He has to stop the execution and start the edit mode again. Just like traditional textual programming, the user has to use the editor to modify the source code, compile it, run it, and eventually gets back to the editor to modify again. I personally think that a good visual programming environment should not make edit and execution modes exclusive. Instead, the designer should use the user's prospect and try to make it truly user friendly within the time and space constraints.

NeatTools provides a non-stop execute mode, and users can turn the edit mode on and off as they wish. This is actually a difficult task even for an experienced programmer, because, each module could start its thread and the following concurrent situations could happen: a user tries to delete a connection but the source module is broadcasting messages through this connection. Or, a user tries to delete a module, but this module is still updating itself through graphics context. It is usually very hard to find bugs inside a multi-threading system, because the bug will not always occur. There are always possibilities for it to occur, but you never know exactly when it will. This makes debugging in multi-threading systems sometimes almost impossible. Thus, how we implement the functionality and make it thread-safe is a most important topic. In NeatTools, the technique I use is to first identify the critical sections inside operation implementation sections. Critical sections are usually operations involving those resources shared by more than one thread. Second, I carefully analyze the critical sections and add exclusive mutex only when it is necessary. Theoretically, this process looks simple, but to identify critical sections, bring out a thread-safe design, and still provide high performance; one needs knowledge and experience in multi-threading systems and concurrency issues.

4.5.5     Identify the Modules

How we identify the module as a functional and reusable object unit is the most important and complicated task. If we liken the NeatTools system to the human body, then the modules are the different organs that work together.

For example, in the previous section, the mouse driver prototype was a small program that performed data acquisition, process, and action mapping. In NeatTools, the desktop provides a visual-programming environment where users can create, manipulate modules, and make connections. We still have to identify and design the module in order to restore the functionality provided in the mouse driver. In the mouse driver's case, we could have COM port, TNG, Calibrator, Viewer, Mouse Simulator, and some logical operation unit to provide the gesture recognition. The module could be a unique entity in the computer like COM ports that represent the physical hardware ports and the software functions related operations like open/close the COM and input/output byte stream, etc. It could be a protocol, like the TNG that receives the byte stream from COM and implements the TNG protocol that extracts the signal information from the byte stream. Or, it could be a calculation unit that provides computational services, like AND, OR, XOR, Calibrator, etc. Identifying modules is important because the requirement specifications and design will use them as the building blocks. Mistakes in properly identifying them will affect the extensibility and maintainability of the data flow network.

4.5.6     Dynamic Ports

Most visual programming environments do not provide dynamic ports for their modules. By dynamic ports, I mean that the input and output ports of a module could be changed in run time. For example, a database module could have several input/output ports for its fields in a database. When a database module is assigned to a different database file, its port configurations (like number, data type, and tags) change dynamically.

4.5.7     Display Feedback Mechanism

NeatTools is meant to be a tool that can apply in HCI (Human Computer Interface) and the tele-medicine field. The display feedback lag time should be minimized when possible. Usually, the display procedure inside a module will involve operations to Windows graphics contexts, which are relatively slow compared to most other non-display operations. In some situations, especially in fine-grained data flow network design, high frequency events may trigger a sequence of display refresh in modules. Sometimes, it could slow down the system a great deal or even hang the system completely.

There are two approaches that could be combined to provide a better solution. First, decrease the overhead of module display. In NeatTools, modules could be resized or overlapped by users. When a module requests a display refresh to itself, a graphics context has to be created and a region has to calculated before the module's update method could be invoked. Region is a logical set of pixels or rectangles that could associate with a graphics context to determine the region that could apply or prohibit the graphics operations. Region can be a simple rectangle or very complex shape because of overlap by several other modules of different dimensions. It is an O(n) operation for a desktop with n modules to calculate the logical region that modules associate with a graphics context. This could be a very expensive operation, especially when the number of modules in a data flow network becomes large. It becomes even worse when a module needs to update itself frequently. To eliminate the region calculation associated with each module display update, I add a region data member inside the module and make the region calculation when it needs to be done like create, move, duplicate, or resize the modules. This change makes the module display refresh an O(1) operation with much less overhead. Second, some repeated display refresh needs to be eliminated when possible. In modern windows systems, there are usually some mechanisms to decrease the need of window redraw. In Microsoft Windows, when an application sends out several invalidation requests to the window system, it will combine the invalidate regions into a bigger region and send out less paint requests back to the application. In X Windows it has the similar mechanism. But, this is not good enough for NeatTools. Because a paint request from the Windows system will cause the desktop to trigger an O(n) calculation to determine which modules need to be updated. To decrease the paint requests from windows system when a module updates itself, it calls a utility method that puts itself into a queue. Later, when the system is idle, a procedure will pick up the queue, delete the repeated request and refresh the modules only once. The module itself has to cooperate with this mechanism so that it should be able to make incremental updates without trouble from missing refresh requests. These two approaches proved to increase the module refresh and increase efficiency by utilizing the CPU idle cycle.

4.5.8     Cyclic Data Flow Network Issues


Figure 24: Cyclic Data Flow Network Issues

In a data flow network with synchronized event dispatching mechanisms, a cyclic data flow network could cause a system to hang by infinite loop. So, instead of avoiding cyclic networks, NeatTools allows a cyclic data flow network to exist without hanging itself. The speed is always the key concern in NeatTools. Thus, how we could propose an approach to detect the cyclic network efficiently and prevent the infinite loop was quite a headache.

In NeatTools, a module notifies the connected modules that an event needs to be processed by invoking the broadcast utility method, so it is true that when a module's broadcast method on a particular port is invoked recursively more than once, a cyclic situation is found. Using this observation, we add a counter that associates with each output port and identify the cycle without extra analysis methods. But, when it comes to multi-threaded environments like NeatTools, this approach will not always work. For example, a module issues a broadcast event and at the same time another module uses a different thread and issues an event that flows through the same broadcast port. The counter that associates with the broadcast port will detect a cyclic situation but actually it isn't. In this case, adding a mutex inside the broadcast method would be enough to prevent the error in cycle detection. But adding the mutex as a default could decrease the parallelism in event broadcasting a great deal. And, depending on the module design, not every port would always have multi-threading events flowing through. Here, I leave it to the module designer's hand to determine where the critical sections are that need the mutex added to avoid concurrent problems. The users who design data flow networks just use the modules and make connections. They don't have to know what is going on under the hood.

4.5.9     Thread and Concurrent Related Issues

Using threads properly could increase the system utilization by eliminating unnecessary busy waiting loops. For example, in the receiving portion of a Socket module, we never know when the next event will come in through the network. The traditional way is to use a busy waiting loop and non-blocking sockets to keep reading on the socket or use the select function to check the socket status. These loops will usually cause busy waiting and waste the CPU cycle. By initiating a thread and using a regular blocking socket to read the socket in the thread, when no data comes in, the socket will block the thread. The application's main thread is still working without having been blocked by the socket read operation. That way, we could maximize the performance without complex implementation and time sharing inside the application and leave the problems to the operating system that provided the threads.

In NeatTools, each module is allowed to have its own threads, yet it depends on the module designer's decisions. But there are some issues that have to be taken care of. For example, deleting a module without destroying the associated threads could lead to a system crash, waste resources, or cause a memory leak. Destroying a working thread without "join thread" operation could cause an event broadcast to be interrupted abnormally and cause problems. So, in the appendix, I dedicate some discussion to focus on the guidelines and tips on using threading in the NeatTools module efficiently and correctly. But, even if module designers follow the guidelines to create, run, join, and destroy the thread properly, deadlock still could happen in some special situations. For example, a module initiates and starts a thread. In this thread, it broadcasts events in a loop. Eventually, one of the events caused the module itself to close. Now the problem comes, when the module's close method is invoked, it will try to use the "waitFor" method to join the thread so the loop inside the thread will have a completed cycle and exit successfully. In this case, the "waitFor" method would never return because the loop is stuck in the event broadcast process, which is stuck in the "waitFor" method itself. Now deadlock happens and the system hangs forever. To prevent this particular situation, a user could use mutex and some extra counter variable to keep track of the possible deadlock. My suggestion to the thread issue is, use it only when necessary and always use it with care.

4.6     Experiments

During the NeatTools' construction stage, I kept improving the functionality by putting in more modules and increasing the usability by providing more services in the desktop area. I kept a working page at http://www.pulsar.org/ej/work.html and put in descriptions and experimental data flow networks with screen dumps. Thus, in parallel to the hardware team's effort on the headset, improving sensor technology, and TNG device, I cooperated with the hardware team by team work on developing a new TNG protocol, new modules, and sample data flow networks for them to learn and design more applications for Eyal and Brooke.


Figure 25: Headsets to pickup gesture signal

After I finished the basic functionality of the mouse driver prototype level, I laid out a network for experimental purposes. At this point, NeatTools could do exactly what the old mouse driver prototype could do, and we could start to make all kinds of different network designs to improve it and make it more useful, not just a fixed, hard coded program.

Figure 26: Mouse driver prototype (1)

Later on, I proposed more prototype networks for Eyal to control the mouse. This time, I used signal gesture to control the mouse direction, movement and mouse button click.

Figure 27: Mouse driver prototype (2)

I laid out another network that would let Eyal control the highlighted column and row of a keyboard module and use it to type characters into other applications like Word, Notepad, etc. By using this network, Eyal was able to type characters into a computer for the first time in his life. Later on, we added an assistant program that could read the words he typed.

Figure 28: Typing utility prototype

We keep improving the software and hardware. For example, the hardware team used photocells a great deal. They are cheap and sensitive to light and very good at picking up the facial expressions signals. But photocells change their signal level when an environment's brightness changes. To accommodate this situation, the team member in the session has to keep calibrating once in a while. Sometimes, it is too much trouble for the family member who is not quite familiar with computer operations. So, the hardware team keeps trying to find something that has a steadier signal output. They tried micro switches and, eventually, came across a special joystick that has a soft rubber at the end of the stick. Eyal could use his mouth to control the joystick very well. Around the same time, Dr. Edward Lipson laid out a very complex and sophisticated data flow network in NeatTools called Joy-Mouse.

Figure 29: Joy-Mouse

In the Joy-Mouse data flow network, a user could precisely position the mouse cursor under the control of a joystick. This can be either a standard computer joystick, or a custom one (e.g. small chin joystick for Eyal) that delivers XY signals to analog inputs of a computer interface module such as TNG-3. The adjustable nonlinear profiles in the quadratic and cubic modes provide precise control of the cursor for small joystick deflections, while allowing rapid movement for large deflections. For more information please refer to the URL: http://www.pulsar.org/neattools/edl/joymouse_docs/JoyMouseManual.html

Dr. Edward Lipson happened to come across a commercial product called FITALY. The small application provides a keyboard interface. A user could use mouse clicks to generate virtually any key stokes. The most important aspect is that the keyboard interface is arranged according to the character frequency analyzed by huge sets of documents, so, it minimizes the mouse movements when trying to use FITALY to generate key stokes.

Figure 30: FITALY keyboard

4.7     Results


Figure 31: Experiment results

By using Joy-Mouse, NeatTools, FITALY keyboard, TNG-3 device, Headsets, and sensors, we started to provide a feasible way for Eyal to surf the web, use applications, and send e-mail to his friends. For Brooke, the hardware team designed all kinds of foot pedals and hand devices. Combined with Joy-Mouse, NeatTools, and TNG-3 device, she could now use her hands and feet to control commercial educational applications, learn from the courses inside applications, and have her progress evaluated.

4.8     Conclusion

In my experiments, NeatTools did provide a powerful and affordable way to improve the life of disabled people like Eyal and Brooke. But, there is always room to improve our approaches. For example, using facial express to type characters by using Joy-Mouse and the FITALY keyboard is still very slow compared to normal people typing by hand. How could we provide a more sophisticated method to map customized facial expressions directly into characters, words, or event sentences? How we could add more AI support in helping them self correct and increase the human computer interface throughput? These are great ideas for future development.

5.     Conclusion

5.1     Future Development

I have worked on NeatTools for around 18 months. Right now, I have already finished most of the core structure and functionality. Scaleable software development like NeatTools usually never comes to an end. There is always plenty of room to make some adjustments and improvements.

5.1.1     Aggregate Data Type

The Aggregate data type is an interesting concept. NeatTools already provides a way to extend new data types and register them into the system. If a user could aggregate different data types into a single complex data type, later on, provided the mechanism to construct nested complex data types visually, it would be very useful for some applications needed to handle several data members at the same time. For example, if we could provide aggregate data types that have X, Y, and Z data members inside and use it as a single 3D point data type, it would increase the readability and efficiency of the data flow network.

5.1.2     Connection Visibility

When data flow networks become more and more complex over time, the connections and modules are all over the desktop area and become very hard to tell apart. Even worse, some connections may overlap partially or even completely, or some other modules may cover them. How we could provide a method to solve this situation did take a lot of effort. Currently, NeatTools has a highlight mechanism, which will highlight the current closest connection or module and display the related information in a status bar. Thus, users can perform some operations on the highlighted connection or module. But it still could not handle the overlap or covered connections very well. I wish I could have an efficient algorithm to arrange all the connections and minimize the overlap between connections, so the connections would always be very clear and easy to read.

5.1.3     Undo/Redo Features

Although undo is a common and rather simple feature in word processors and spreadsheets, undo in a graphical programming language is a very sophisticated and challenging utility. An extensive, versatile undo/redo feature in NeatTools will take a lot of time to implement. This feature is extremely important to the NeatTools user. I wish I were able to add this feature in the near future.

5.1.4     Network Support Modules

Currently, the Socket and ServerSocket modules already support the TCP/IP protocol. To enable NeatTools users with the full power of processing the vast information from the web, FTP sites, and news servers, we will develop WWW, FTP, and NEWS modules to provide the service and serve as information sources that could be processed and visualized by NeatTools.

5.1.5     Authentication Modules

To enable a secure visual-programming environment in a network/distributed environment, we will need authentication modules that facilitate public-key encryption to protect the network communication between NeatTools from tamper activities.

5.1.6     Visualization Modules

Visualization modules always play an important role in human computer interface tools. As a human computer interface visual-programming tool, NeatTools will need more visualization modules in the future. Currently, we have some standard 1D viewer, 2D viewer, sliders, LED, etc. The 3D viewer, Space-time viewer, and 3D surface visualization modules are yet to come. Those visualization modules will provide key functionality to facilitate future human computer interface activities.

5.1.7     Artistic Modules

Currently, I have designed the modules emphasizing on their functionality. So, sometimes, the appearance is not so appealing. In the future, we could consider adding more artistic modules in NeatTools. It could provide functionality as well as the overall look and feel. About the artistic aspect, we would need people with computer graphics training and also a good sense of artistic design. A lot of commercial applications including LabView do a very good job on this aspect. NeatTools needs to work on this more.

5.2     Conclusion

The focus of this thesis is my work on proposing a new architecture of fine-grained data flow network visual programming environment - NeatTools. I have detailed descriptions about module abstraction, which forms the module event broadcast model in NeatTools. I also identify and specify some of the related systems and ideas. During the experiments for Eyal and Brooke, several key issues about building a visual programming environment were explored and discussed.

Our major findings are:

(1)      The same idea could have variant performance based on different design concepts and implementation architectures. For example, both AVS and NeatTools are provided as data flow visual programming environments. The AVS kernel provided the centralized control over the inter-modular communication, and hence serialized parallel channels. On the contrary, in NeatTools, different modules could issue parallel events under different threads and become a distributed event broadcast system. So, NeatTools has higher performance in event broadcasting and could meet the requirements of a fine-grain data-flow visual-programming environment.

(2)      The coarse-grained data flow model is well suited for combining subsystems and providing integrative solutions. But, all the fine-grain operations and functionality have to implement inside the modules and can not be utilized by users directly through the visual programming environment. It could reduce the function reusability and increase the implement redundancy. In this sense, a fine-grained data flow model would be a better choice for a designer to identify the reusable objects as the building blocks.

(3)      Use of commercial application frameworks like MFC to construct windows applications seems to be obvious for managing the complexity of a low level API like Win32 API. But it could cause an application to become native to a specific platform and become very hard to port to other platform like X Windows. Without modifying the application itself, one could build a simulated Win32 layer on top of UNIX and X Lib (and MOTIF) to provide the cross platform solution (Like Win/U by Bristol Technology). But this solution could introduce extra layers and decrease the application's performance. If we could add a unifying thin layer between a Windows operating system and an application, cross platform and high performance could be achieved at the same time.

(4)      To increase the user development speed, distinct and exclusive modes like edit and execution are not desirable; it is necessary in traditional textual programming environment, because the source code needs to be compiled before it can be executed. In a visual programming environment, we could keep the data flow network in constant running mode without interrupts by edit mode operations. Also, we could maintain maximum performance and solve all possible concurrent problems. Then, users could design, execute, and debug at the same time and increase the development throughput.

6.     Appendix

6.1     NeatTools Reference Manual

6.1.1     NeatTools Module Specification:

 

Name: NOTObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise NOT operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) NOT operation. When it is 16, it will become the 16 bits wise NOT operation.

Name: ANDObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise AND operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) AND operation. When it is 16, it will become the 16 bits wise AND operation.

Name: ORObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise OR operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) OR operation. When it is 16, it will become the 16 bits wise OR operation.

Name: XORObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise XOR operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) XOR operation. When it is 16, it will become the 16 bits wise XOR operation.

Name: GreaterThanObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is larger "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).

Name: GreaterEqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is greater or equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).

Name: EqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).

Name: NotEqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is not equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).

Name: AddObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the sum of all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: MultiplyObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Multiply all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: SubstractObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" - "input-1" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: DivideObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" / "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: AbsObj

Input: "input" integer port on the left with maximum one connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate ABS("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

 

Name: PowObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" ^ "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: MaxObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Choose the maximum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: MinObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Choose the minimum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: RemainObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" % "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

Name: RandomObj

Input: "control(logical)" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will generate a random integer number and broadcast the result value to "output" port.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).

Name: ControlObj

Input: "N(1-512)" and "control(logical)" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorphic ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" polymorphic port on the right edge with unlimited connections.

Function: "N(1-512)" will determine the number of input-output pair. When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will connect the input-output pairs. So when there are input events presented on say "in-k", this module will transmit them into the corresponding "out-k" output port.

Properties: "moduleColor" set the background color. "N(1-512)" set the number of the input-output pair.

Name: SampleObj

Input: "N(1-512)" and "control(logical)" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorphic ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" polymorphic port on the right edge with unlimited connections.

Function: "N(1-512)" will determine the number of input-output pair. When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will sample the current input value from input port say "in-k" and transmit into the corresponding "out-k" output port.

Properties: "moduleColor" set the background color. "N(1-512)" set the number of the input-output pair.

Name: PulseObj

Input: "input(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving an input event, this module will generate an true-false event pair to the "output" port.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).

Name: DelayObj

Input: "input(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving an input event, this module will hold the value and broadcast the value when next input event comes in.

Properties: "moduleColor" set the background color.

Name: AccumulatorObj

Input: "enable(logical)" integer port on the top edge and "clock(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving a raise signal (from false to true) event on "clock" port and current value of "enable" port is true, this module will increase the internal integer accumulator and broadcast the value to the "output" port. If receive a drop signal (from true to false) event on "enable" port, the internal integer accumulator will reset to zero and broadcast the value to the "output" port.

Properties: "moduleColor" set the background color.

Name: MultiplexerObj

Input: "N(2-512)" and "select" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorphic ports on the left edge with maximum one connection.

Output: "output" polymorphic port on the right edge with unlimited connections.

Function: "N(2-512)" will determine the number of input port. When receiving an input event on "select" port say k, this module will connect the selected "in-k" input port to "output" port. So when there are input events presented on "in-k" port, this module will transmit them into the "output" port.

Properties: "moduleColor" set the background color. "N(2-512)" set the number of the input port. "select (0-(N-1))" set the current selected input port.

Name: DeMultiplexerObj

Input: "N(2-512)" and "select" integer port on the top edge and "input" polymorphic ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ...  "out-(N-1)" polymorphic port on the right edge with unlimited connections.

Function: "N(2-512)" will determine the number of output port. When receiving an input event on "select" port say k, this module will connect the selected "input" input port to "out-k" port. So when there are input events presented on "input" port, this module will transmit them into the "out-k" port.

Properties: "moduleColor" set the background color. "N(2-512)" set the number of the output port. "select (0-(N-1))" set the current selected output port.

Name: EncoderObj

Input: "N(2-32)" integer port on the top edge and "in-0", "in-1"