Project Summary for NSF Information Technology Research (ITR) Program

(Information Technology Education and Workforce, and Information Management areas)

Computer Science Curriculum and the
Next Generation of Education Technologies

Principal Investigator: Geoffrey Fox (Florida State University)

Co-Investigators: Willie Brown (Jackson State University), Chris Lacher (Florida State University), Sara Stoecklin (Florida A and M University), Joe Thompson (Mississippi State University)

Senior Personnel: Larry Dennis (Florida State University), Ian Douglas(Florida State University), Peter Dragovitsch (Florida State University), Roscoe Giles (Boston University), Carole Hayes(Florida State University), William Lupton (Morgan State University), Joseph Monroe (North Carolina A&T State University), James Turner(Florida State University) Nancy McCracken (Syracuse University)

We present a proposal for innovative research into both the methodology and technology needed for new models of computer science education that will be accessible to a broad range of learners. The team consists of Florida A and M, Florida State, Jackson State, Mississippi State, the NSF Education Outreach and Training effort in the PACI program and several other HBCUs.

Rapid advances in computer technology require computer science curriculum changes to best prepare students for jobs in business, academia and government. These advances further allow new types of interactive courseware, reusable learning object modules, new learning environments and new business models for educational infrastructure. This proposal weaves these themes together to develop prototype undergraduate computer and computational science curriculum learning modules and conduct research in the area of distance and distributed learning environments deployable within the next few years. While our focus is on the particular needs of Historically Black Colleges and Universities (HBCU), dissemination of modules is not limited to a particular population. In particular as a second testbed we will use the existing FSU distance education activity aimed at flexible education for the life-long learner. We will research architectures that allow modular courseware developed by different authors and different authoring strategies. Further we assume that learning environments should allow integration of capabilities from multiple academic and commercial sources.

The major components of the project will be:

· Develop interactive computer science courseware reusable object learning modules exploiting the best educational technologies and preparing tomorrow's undergraduates for careers involving computers. These courseware modules will be integrated into existing computer, computational and information science curriculum course sequences;

· Research in and prototype development of a next generation learning environments exploiting the best academic and commercial ideas in both the education specific and general information areas. This environment will support synchronous, asynchronous and interactive learning models;

· Deliver to a broad-based student body, the new course modules developed by teachers from the participating universities;

· Assess and evaluate both the new curriculum material and the information technology used to prepare and deliver it.

A major result will be a networked computer and computational science courseware module delivery system. These courseware modules presented over the Internet will supplement on-campus CS curricula courses at HBCUs and other major CS departments around the country. This infrastructure will build on experience gained from the current successful delivery system used at Syracuse with CS courses taught to other sites including Jackson State (an HBCU). Jackson State now uses this delivery technology to teach their own CS courses at Morgan State. This effort is having a significant effect on the pipeline of minority CS graduates, enhancing the quality of their education and also serving to increase the attraction of a computer science career. We will expand this successful activity by providing the delivery of learning modules to other HBCUs – initially Morgan State and North Carolina A and T, and Elizabeth City, Morehouse and Spelman.

We will adopt a well designed curriculum model built in terms of reusable modules stored in a common repository that will be the basis of both our dissemination and resource used by our Web-based educational system. Our approach to education technology will be built around the concept of a collaborative portal with shared events supported in both synchronous and asynchronous mode. We will develop a new system using ideas and components from previous commercial and academic systems such as Syracuse's synchronous TangoInteractive system developed over the last three years. We will also exploit Florida State's experience using the commercial Blackboard technology and a recent complete evaluation of current practice from Mississippi State. We will use a distributed object framework such as Ninja from UCB or E-Speak from Hewlett-Packard and systematic use of XML metadata conforming to community standards as they are developed. A key requirement and major research issue will the ability to support course modules and tools from multiple sources interoperating with common services and interfaces.

This proposal forms a unique partnership consisting of HBCUs, research institutions, international research centers, and a selected number of Florida community colleges. The network described in this proposal provides an overall organizational structure, which will leverage existing research expertise among participating institutions, assist in the development of a pool of minority researchers, and facilitate joint university cooperation and involvement at a high level.

1: Motivation: Workforce, Technology and Education

The continued and growing need for computer professionals is documented in many formal and informal ways. Data from the U.S. Bureau of Labor Statistics suggest the need for a 100% increase in the production of these professionals and the figure shows this in another way as the expected growth in shortfall [74,78,95]. It is clear that the number of graduates produced by the nation’s universities will be insufficient to meet this demand and we already see an influx of companies hiring non-US citizens, who are ready and willing to fill these jobs. Additionally many companies are hiring persons with scientific degrees in other disciplines (math, biology, statistics, etc.) and training them in abbreviated fashion to fill computing jobs. NSF Science Resources Studies, the National Center for Education Statistics and the Commission on Professionals in Science and Technology have documented such trends and the latter has in particular highlighted a serious deficiency in the number of minority computing professionals [19]. We suggest that existing universities can meet this need for computer science graduates by turning to distance education.

It appears that traditional approaches are not adequately addressing these trends and in this proposal we will research novel approaches to computer science education that will both increase the quality of the learning environment and allow the increase of graduating students needed by the nation. The products of the proposed work will be both new computer science curriculum and the development of new reusable computer science learning object modules development and the assessment of new technology for learning environments. There has been a rapid profusion of commercial training efforts in this arena [24] but we will focus on higher education courses, which have been proven to be more effective pedagogical approach for producing students with lasting knowledge. We have chosen two distinct and important student bodies as testbeds for our curriculum: firstly a network of historically black colleges (HBCU) led by project partners JSU and FAMU who have already had substantial success in internet based curriculum. Secondly the state of Florida represents one of the fastest growing states with significant large and small computer-based businesses and a clear need for flexible lifelong learning. Here the second major project partner is the FSU Office of Distance and Distributed Learning (ODDL) with institutional responsibility in this area and a new computer science curriculum as a major initial thrust.

Teaching computer science is particularly challenging as the growing student interest is coupled with increasing difficulty in hiring good faculty and the need for constantly updating courses and whole curriculum to maintain relevance in a technology cauldron stirred with Internet time. Our testbeds are set up as institutional networks shown in the center of fig. 2, that naturally allow faculty, mentors and students to participate in the learning process and so increase the pool of qualified and current teachers. Course content changing with Internet time implies substantially more faculty involvement in the continuing evaluation and upgrading of the curriculum. This accentuates the need for quality learning environments that scale to many more students than a traditional classroom. This naturally suggests Internet based distance education supported by a hierarchical network of teaching assistants, mentors and faculty.

This strategy is illustrated in fig. 2, which shows our proposed collaborative network of universities designing and developing shared courseware placed in a repository managed by a modern distributed object system. We expect that each network member would integrate the shared courseware repository into separate learning instances. These are particular course programs leading to degrees meeting the special needs of their learners and other stakeholders. This delivery is supported as necessary by mentors and teachers who may or may not be part of the university responsible (in terms of finally awarding the accredited degree) for this program.

The technology component of our project will research and deploy a mix of academic and commercial capabilities to enable such a learning paradigm. Several approaches to web-based (distance) education have been developed and applied with some success. These include the largely asynchronous database linked commercial Blackboard system being deployed by FSU and the synchronous collaboration based courses delivered over last 3 years between Syracuse (Fox) JSU and other HBCUs [8,9,76]. Looking to the future, distance education will be a key part of the efforts to increase the efficiency of higher education and to adapt curricula to the changing demands of modern society.

There are many possible models for web-based education but we suggest that there are no clear “winners” for today what we see is warped by institutional legacies and immature technology. Synchronous instruction comes with an ongoing high price tag that cannot be reduced due to the human factor (faculty) and his/her limited availability in time. Asynchronous education has a higher up front cost, which is a difficulty in rapidly varying curriculum and where authoring technology is still changing rapidly. We see the needs for unified systems supporting different interactivity models and further that this choice will be customizable to the individual learner. We anticipate that five years from now the seemingly oxymoron of providing individualized education in the mass production learning environment of a virtual university should become reality. The computer science research component of our proposal will develop a framework built around collaborative portal technology that will support these key characteristics of unification of interaction paradigms and the customizability for each learner. This framework must inevitably support a variety of tools coming from a mix of academic and commercial sources. Further the technology decisions will be structured as relatively short 6-12 month modular projects for the adjustment to a technology and tool environment moving with Internet time.

As we innovate both delivery technology and computer science curriculum, the project is fundamentally centered on its two learning testbeds described in Sec. 2 and the assessment activity of Sec. 3.3 to evaluate both technology and curriculum. Our approach to curriculum design is described in Secs. 3.1 and 3.2. The computer science contributions of this proposal are to both “Education and Workforce” and research in the distributed system technology to support a virtual university. The latter is described in Sec. 4 together with a discussion of important national standards activities within which we will work. while In Sec. 5, we present our management and budget issues are in Sec. 5 and comments on the team in Sec. 6.plans for management and research and describe our dissemination activities. Sec. 6 summarizes the capabilities of the key participating institutions. In the International appendix, we describe three existing activities in Africa, China, Europe and South America which will be very synergistic with this project and derive mutual benefit from visitors programs and the exchange of course modules and technology.

2: Collaborative University Network

2.1: HBCU Computer and Computational Science Testbeds

The project is centered on computer science education in two major testbeds. The largest will be a network of HBCU universities starting with our computer science partners JSU, FAMU, Morgan State, North Carolina A & T. These are joined by HBCUs Elizabeth City, Morehouse and Spelman for computational science. An essential idea behind our approach is the scaling of quality educational material by using technology that supports dissemination to many students and simultaneous training of teachers, mentors and assistants. We will implement this by the exchange of material between the participating universities; a concept successfully tested by Syracuse, JSU and Morgan State [8,9]. The next steps in this process are given in more detail in Sec. 5.2 and include:

1) Identify similarities among curriculum and course content characteristics that allow categorization of courses and places where courses can be shared.

2) Identify candidate course delivery mechanisms.

3) Provide adequate infrastructure at participating colleges/universities.

4) Deliver similar course content with different technologies using flexible multi-source framework of Sec. 4.

5) Evaluate results using assessment process of section 3.3. This will lead to an understanding for each of several categories of courses, which technologies/software tools/environments are best suited for course delivery, in both distance education and the resident classroom

Further HBCU partners in existing programs with which we are associated, will be used to grow the network in future years. This includes DoD PET (Programming Environment and Training) partners at ARL, ASC, ERDC and NAVO: Alcorn State University, Central State University, Clark Atlanta University, Grambling State University, Southern University, Tennessee State University. The NASA Minority University - Space Interdisciplinary Network (MU-SPIN [72]) Network Resource and Training Sites (NRTS) bring City College of New York (CCNY), Elizabeth City State University, Prairie View A & M University, South Carolina State University, Tennessee State University, University of Texas at El Paso. The Army High Performance Computing Research Center involves Clark Atlanta University, and Howard University. For the first two years, we anticipate that the initial 7 HBCUs in computer science and computational science will pioneer the collaborative network and once this is successful, we will judiciously expand the project using these other colleges for which partnerships are already in place. The organization of these partners will be the responsibility of JSU, which has recognized that Web-based distance education technologies offer tremendous potential benefits to the HBCU/MI community, including curricular enhancement, sharing of limited resources, and collaborative teaching/learning. JSU has already developed a university wide strategic plan for distance education and training that we will leverage in this NSF ITR proposal. While this effort involves multiple universities, many of these universities have existing experiences with this type of collaboration and others share a close proximity to one another. This proposal builds on on-going strong collaborative efforts and poses no problems with close working relationships.

A successful collaborative university network requires that the partners have an adequate infrastructure to support the innovative course development and delivery. This infrastructure includes 1) suitable physical classroom facilities, 2) a reliable and sufficient connection to the Internet, and 3) on-site human resources. JSU has gained considerable expertise and experience with respect to what is needed, and effective procedures to overcome the barriers to implementation. This gives us a heads start on the design, planning, procurement, and installation of required equipment and connections at selected partnership institutions. The project will establish the necessary process and infrastructure for the training of collaborating faculty and staff. We will initiate this with a fully equipped, and staffed, teaching and learning laboratory at JSU that will allow 1) collaborative course development and 2) cost-effective local and remote instructional training with collaborating schools. Such training and support is essential to the success of this project. We intend to build upon this foundation and develop a national resource for technologies supporting electronic delivery of education and training, which will facilitate inclusion of, and broaden the participation of, underrepresented groups in information technology careers. Note we do not intend to supply significant network infrastructure as part of this proposal as NSF already has in place efforts in this area. There is for example the Educause/NSF PACI EOT Advanced Networking Project with Minority -Serving Institutions (AN-MSI) grant. Roscoe Giles as joint leader of the PACI EOT will ensure this synergy. We hope that membership in our network will encourage universities to upgrade their IT infrastructure, which will of course have far reaching benefits outside our project.

Faculty and staff in the network of universities, will develop course module content, receive courses from other institutions, and deliver courses to partner schools. A result of this process will be:

1) Well-defined principles for course module development and delivery.

2) A coalition of HBCU/MI colleges/universities equipped to develop, deliver and receive courses.

3) A large number of faculty, staff, and students who are more IT literate.

4) A large number of students (both students and teaching assistants) better trained for IT careers.

In addition to being an existing network of collaborators and representing a highly desirable target population, the HBCUs bring another unique advantage to the table: Their historical mission has been to educate marginalized people and empower them both to enter the mainstream and/or become leaders of the community. A special element of their programs has been the special attention paid to developing students and to the relation of their students to society. Arguably, the rapid onset of the eWorld and the consequent need for and shortage of IT workers has put much of America's educational institutions in a similar position: (a) many people at the margins needing to be educated to effectively participate in and lead the IT revolution; (b) need for attention to student development in relation to the larger society; (c) much of the academic content is generated outside the institution and imported and adopted. Thus the choice to work with HBCUs represents a mechanism for prototyping and developing best practices that will apply to the country as a whole. In this sense, HBCUs are leading the development of new curriculum and associated required technology that will generalize to major communities nation-wide.

In the next section we describe our second testbed where the network consisting of FSU and Florida community colleges is already in place. Here we will use project courseware repository, technology and methodology and see how the different student demographic and more tightly coupled organization affect the success of the approach depicted in fig. 2.

2.2 Flexible Lifelong Learning Testbed

2.2.1: Introduction

Florida State University (FSU) in engaged in several university wide initiatives that are synergistic with this proposal. The long-term goal of this project is to provide high-quality courses and degree programs to Florida Community College Students, FSU students in residence, students at FSU's international branch campuses and professionals. FSU is currently establishing the personnel, procedural and technological infrastructure necessary to support these activities. FSU's Department of Computer Science, working in cooperation with the State Community College System, has developed a new 2+2 program that allows student with the equivalent of a 2-year Florida Associate of Arts degree to complete a Bachelors degree via distance learning. Working with this institutional effort gives us access to professional infrastructure in areas like assessment and technology support. Further it gives us a very different student body to work with – typically more mature students and often with a daytime job. Any learning environment that it is broadly useful must support this typical lifelong learning scenario. FSU has designed its approach to distance education to give equal educational opportunities for residential and distance learners. This allows us to tie the lifelong learning testbed to traditional undergraduate education as FSU will contribute to and access the shared courseware repository for both classes of students.

2.2.2: 3 Layer Learning Model

FSU has designed a three-layer model of delivery, which is very consistent with the approach we intend to use here and used in Syracuse-JSU distance lectures. FSU’s system is adapted from two proven models of "middle-layer mediated" instruction: the large-lecture class, run by a senior faculty member and mediated with teaching assistants (TAs); and the tutor system developed over the last 30 years by the Open University (OU) in Great Britain. Unlike the paper and British Post system of the OU, however, our system uses the full power of the Internet to facilitate rapid and timely interactions among students, mediators, and faculty. The mediators in this instance are called mentors. Mentors are recruited from a pool of applicants drawing from Community College faculty and qualified private sector individuals. The lead faculty member and the academic unit offering the program do selection and appointment of mentors from the candidate pool. Creation and management of the mentor candidate pool are coordinated centrally by ODDL.

FSU's experience so far is that this 3-layer model is both highly effective in teaching students and efficient with faculty time. It can be used in both the classical (large lecture, TA-mediated) and distance (Internet-supported, mentor-mediated) modes. FSU is now adapting this model to its growing list of branch campuses and international centers described in the international section of the proposal. Without making rules, a culture of communication has been established in which the mentor is the student's first point of contact. By handling most communications locally in the hierarchy, and keeping the student/mentor ratio low, this system has alleviated the problem of communication overload that has been typical of less organized, email-based attempts at Internet-supported distance learning. We will use these lessons in the HBCU network and an important result of this project will not only be such methodologies but also the technology to support them.

2.2.3: Enabling Infrastructure

There are several key features of FSU's effort that help create an environment in which we can test our technologies, resources and ideas in a wide variety of situations and get participation from students who have a diverse set of goals, interests and skills. In particular FSU's effort solves problems and provides resources that would not be possible within the scope of this project. For example, numerous institutional obstacles, such as requiring students to come to Tallahassee to get a picture student ID card or to get student loans are being removed. Additionally, a 24 hour-a-day, 7 days a week, online help desk and phone support system is being created to assist distant students with computer problems. FSU is establishing a network for recruiting and training the mentors discussed above. A high-quality cadre of mentors to assist students locally makes it possible to test the scaling of our efforts with large numbers of students. Finally, FSU is establishing the computer hardware and software infrastructure required to support large-scale delivery of courses.

Several of the people on this proposal (Lacher, Dennis and Dragovitsch) are actively involved in directing this effort. Lacher is Director of the Office of Distributed and Distance Learning, which is helping faculty create the online curriculum. Dennis is directing effort to develop tools to help faculty whose instructional needs are not being met by the standard online environment (CourseInfo). Dragovitsch is organizing faculty from across the campus to serve as an advisory team for this project. Fox has just been appointed chief technologist for ODDL, which quantifies the University commitment to integrate “what works today” with an innovative vision of the future.

2.2.4: Learning Modules and the Shared Courseware Repository

At the heart of the FSU delivery model is a set of core curriculum components. It has become increasingly clear that there is considerable effort and expense involved in developing reusable and retargetable activities and materials. This effort is repaid, in part, by the inherent accumulation effect, wherein the components are saved from one offering to the next, continuously improved over time, and added to by a variety of contributors. Nevertheless, the effort and expense are such that the sharing of components, across time, across courses, across programs, and across universities, would be ideal. The essentially standardless system currently in use (at FSU and elsewhere) produces some excellent materials, but re-use requires person-to-person interactions and intimate knowledge of how the materials work. What is needed is an organizing and unifying system of shareable learning objects that facilitates the use and recombination of components with only external knowledge of these components.

The collaborative unifying system of courseware development and re-use proposed herein exactly meets these needs. All three uses of the middle-layer-mediated delivery model (classical, 2+2, and branch campus) are learning instances (testbeds) in the sense of Figure 2. FSU will make significant use of the shareable courseware repository as well as contribute to the repository. Value will be added for FSU as well as all other users of the repository, resulting in both increased efficiency and higher quality of computer science programs.

The computer science/software engineering curriculum re-design underway at FSU is built on several organizing themes. There is a breadth-first introduction, in which most of the important curriculum threads are initiated. Object-oriented programming is emphasized. Analysis and design (beginning with object-oriented) are taught early and integrated into the rest of the curriculum. And a systems view is taken throughout. Of course, the process is fully informed by national standards (ACM, IEEE), the research strengths of the department, and the consumer community (students and employers). The detailed design and implementation of this new curriculum is taking place over a four year period beginning in Fall 1999. The new coursware already created will be revised for the evolving repository standards, and the courseware developed in the future will be written to these standards.

How such a unifying system can be best designed and implemented is part of the research of this proposal. But the top down system of standards and interfaces must also be open and inclusive, so that bottom up, grass roots contributions are encouraged. The standards framework provides organization to the creative process.

2.3 Authoring of Curriculum

The course material will be primarily aimed at undergraduate computer and computational science students but we will include both middle/high school and graduate level courses where we have success in the past [58]. We will develop (and use pre-existing) interactive material (such as Java applets) and develop common subject specific resources such as quizzes and glossaries. As described in Sec. 4, a major challenge will be to ensure that we have identified the correct places to define standards (in XML). Further we must establish the happy compromise between total freedom in choice of authoring tools and the restrictions imposed by the capabilities of a realistic system framework. For instance the collaboration and assessment services will support some methodologies (e.g. Java and HTML/XML) better than other specialized authoring formats for which the internal event structure and document object model is either unknown or not in accordance with standards like those of the W3C [100].

3: Learning Framework

3.1: Model for curriculum development and the learning object repository

Curriculum models for computer science are developed in a number of ways. A systematic approach to curriculum development would identify who are the stakeholders in the final product of the curriculum, the graduated student, and determine the requirements of those stakeholders. Stakeholders may include industry, government [84], graduate research institutions, and funding providers. As shown in fig. 3 a well-designed curriculum is likely to be influenced by a number of sources including prospective employers, recommendations from professional bodies (e.g. ACM[1]), the internal faculty, government standards, and general commentaries on curriculum matters by external commentators. External commentators could include advisors, curriculum design experts, expert teaching and research faculty, and those who make general statements on curriculum matters in professional publications. Research influencing curriculum development should not only include computer science research, which provides the direction for future educational needs, but also educational (e.g. [49]) and skills research (e.g., [29])

A top-down approach to curriculum and course design is generally encouraged by most experts where the high level learning outcomes are specified for the curriculum [23,70]. Once the desired high-level learning outcomes have been agreed upon, they will be refined into more specific competencies and courses emerge from assembly of related learning objectives. Educational researchers have developed techniques to assist in this process, e.g. [21,27]. Specific learning outcomes should drive the selection and development of learning resources, technologies that mediate the educational experience, and assessment.

This is an idealized model of curriculum development and is seldom completely applied. Often curricula are developed with little reference to outside sources. The curriculum is a result of compromises between the views of internal faculty as to what is appropriate to teach. Many curriculum developers approach the problem as one of identifying courses rather than identifying desired learning outcomes. If we take the analogy with software engineering this is akin to identifying the sub-system architecture prior to determining the systems requirements. Specific learning outcomes, if they are articulated, are derived in a bottom-up design process from the chosen learning materials (usually textbooks).

The systematic top-down process (instructional systems approach) and the informal bottom-up process (traditional reliance upon existing faculty expertise) are two extremes and most curriculum development falls between these. As new curricula are developed or existing curricula revised (a frequent occurrence in computer science), there has been a trend towards a more systematic approach, with accreditation and review processes expecting specified learning outcomes and clear rationales for design choices.

Course developers are often constrained in the learning materials available, most especially in the rapid deployment world of computer technologies. The traditional learning tool has been the textbook, which attempts to cover the learning requirements for a whole course. A textbook is seldom an optimal solution for a course developed using a top-down model, unless it was written to meet a specific need recognized by an expert or experts. A textbook may miss some important learning outcomes for a course or be a poor tool in facilitating others, it often does not provide assessment tools or support different learning styles [50,66]. To supplement the textbook, course developers frequently have to design or obtain a great deal of additional material, e.g. notes, diagrams, animations, assignments, tutorials, and computer-aided learning modules.

It is possible to seek existing materials on the Internet and elsewhere; however this is a difficult process due to the differing standards of description used for materials. Often materials must be downloaded and examined before a determination can be made regarding its efficacy in meeting the course needs. This source suffers the same problem as textbooks in being large packages, which are often only in part useful. When course developers put effort into developing their own learning materials for a particular course the benefits of the resulting material are seldom made available beyond the target course.

This project will facilitate a systematic approach to curriculum design by providing learning materials of sufficient granularity to address specific learning outcomes. It will facilitate access to learning objects with attached meta-data through the Internet accessible courseware repository of Fig. 2. This general model underlies the national standards activities described in Sec. 4.2 and two well-known examples are EOE and MERLOT [28,69]. A key element of the learning object’s meta-data will be the specific learning outcomes and objectives that the object addresses.

In some perspectives, the concept of a learning object is restricted to a unit of computer-aided learning. In our perspective a learning object is any self-contained learning resource that is appropriately tagged according to meta-data standards* and is locatable via metadata indexing and searching services. The key defining feature is not the delivery technology, but the fact that it addresses specific learning outcomes. Thus a learning object may be a Java applet that contains an interactive simulation of a particular concept in operation, a collection of bibliographic citations, or it may be a text document describing an interactive group exercise that can be carried out in a classroom. It is anticipated that the majority of objects will facilitate asynchronous learning given the general trend towards distributed and distance education.

To envisage how the shared courseware repository of fig. 2, will work consider the following scenario: Professor Smith at the Newtown University is developing a new course in “Systems Analysis and Design using UML”. This course is to be added to the undergraduate program in computer science. The professor has identified a number of specific learning objectives, including the following examples.

By the end of this course students should be able to:

“provide a critique of a given sequence diagram”

“convert a class diagram into C++ code”

Professor Smith selects the web reference for the learning object repository, he selects search and enters the keywords “ UML “ and “sequence”. The search results in the display of several learning objectives related to these keywords, one of which,” provide an analysis of a sequence diagram”, sounds similar to what he is looking for. Selecting the identified objective results in the display of a list of learning objects aimed at achieving this learning outcome, it will also display associated assessment objects. Selecting on each object name will display its detailed meta-data. Included in the metadata would be such information as type of learning object (e.g. whether it is instructions for a tutorial exercise or a Java applet containing interactive practice exercise), technology requirements (e.g. requires Internet Explorer version 4 or later), peer reviews of the object’s quality, student feedback on their experience using the objects, the learning model applied*. After selecting one of the objects, Professor Smith then enters the second objective, this time there are no associated learning objectives. The systems asks him if he wants to record this as an unfulfilled need, he selects yes and the learning objective is recorded as one where a need for learning objects exist. Professor Smith does a search for unfulfilled needs using “UML” as a keyword; this results in a list of several learning outcomes that have been entered by other professors. Professor Smith notes that a small computer-aided learning object he recently developed could fulfill one of the outcomes. He selects ‘submit learning object’ and is then given a form to fill in the standard meta-data, after doing this he is able to submit his object. Once submitted an email is automatically sent to all those professors who have registered an interest in this learning objective.

In this way the courseware repository of fig. 2, will fill with a variety of learning objects, using a variety of media and technologies, and supporting a variety of learning styles. The repository enables and assists the developers of learning objects to identify areas in which to concentrate their efforts, i.e. areas where learning objectives have no learning objects and areas where there is a need for learning objects supporting different learning models, or newer technologies. It is also possible that existing learning materials can be easily adapted to the learning object model by sectioning material into object sized units and creating the required meta-data. The repository enables users of learning objects to have a common frame of reference when looking for learning materials to suit their specific requirements. The object meta-data allows users to greatly increase the efficiency of their search and evaluation process when building a collection of learning materials for a course.

3.2: Learning Theory, Models, and Styles

Learning may be defined as a change in performance that comes about as a result of the learner’s interaction with the environment. Theories of learning describe just how this might occur. The major theories of learning are behaviorism [11,87,99], cognitivism [23,41,44], and constructivism [7,13,41,98]. Behaviorism simply links learning with changes in observable behavior; internal mental processes are not emphasized in this model. Cognitivism focuses on the mental processes that mediate learning and bridge to the observable behaviors that follow the learning intervention. Constructivism focuses on student engagement in meaningful experiences from which relevant learning is derived [26]. Consequently, instructional activities are based on curricula that range from very concrete to very abstract based on these theories, as appropriate for the learner and the subject matter.

Learning styles are based on personal preferences or capacities that determine how an individual relates to the environment. Seven types of “intelligence” have been described and learning theorists urge that attention be paid to all of these capacities in design and development of instructional activities [42]. Perceptual preferences and strengths include sensing gateways, that is, auditory, visual, tactile, and kinesthetic [86]. A relationship between the continuum from kinesthetic to auditory and concrete to abstract is relevant in constructing learning experiences. It is usually the case that as the learner matures, reliance upon kinesthetic (concrete experience) learning decreases. However, it is important to be aware of the array of modes of sensing and consider the appropriate application of methods of instructional activity design to the intended learning outcomes. For example, while abstract conceptualization and metacognition are advanced (mature) learning skills, it is altogether inappropriate to rely upon one’s cognitive grasp of CPR in the training process for emergency medical personnel. A strict behaviorist approach is the only valid method of ensuring effective mastery of CPR techniques.

Effective development and identification of learning objects for our computer science curriculum will be related to principles of learning theory, learning style, and instructional models. The variety of learning objects encouraged by this project in itself guarantees coverage of instruction models. The dynamic design allows for ongoing growth and revision to the repository in response to instructor and learner needs. Continuous improvement is, therefore, inherent in the repository design.

Well-designed instructional activities motivate learner interest, present new content, involve the learner in practice and application, assess understanding, and then proceed to the next learning objective [41]. In cognitivism, this process is described by the building upon an existing schema or mental structure through which an individual interprets the environment. Schemata develop and converge to alter the student’s cognitive and affective domains and result in mastery and expertise, i.e., learning. Methods for presenting instructional experiences that building within and upon each other can include programmed instruction [52,91], discovery learning based on real problems and situations [13], cooperative learning [61], drill and practice, expository learning, inquiry-based learning [77], simulations, as well as multiple technologies for conveying these experiences. Objects may be text-based or CAI, and make use of a variety of media, both projected and non-projected [51], such as audio and/or video. Instructional design principles incorporate prerequisite skills and knowledge, learning objectives for the new instruction, methods of application of new learning, and assessment of content or skill mastery [23]. The instructor or learning facilitator will be able to choose and sequence objects appropriately by searching a standardized index of meta-tagged objects. Objects within the repository will support development of formal credit coursework, certificate programs, and just-in-time learning for training and continuing education purposes. In other words, the flexibility and variety of learning objects can satisfy instructor and learner needs in any instance of instructional delivery.

This project does not aim to research these issues of education research but rather to ensure that our work is in accord with best practices in this field. This will be ensured by ongoing interactions with the Learning System Institute LSI at FSU (with which project partner Ian Douglas has a joint appointment) and EOT PACI partners including the CILT Center for Innovative Learning Technologies [16].

3.3 Assessment Plan

Essential to quality assurance will be guidance and confirmation of adherence to principles of good practice. It is assumed that institutions submitting objects for inclusion to the repository are accredited by a nationally recognized accrediting body and that the objects submitted have been appropriately reviewed by discipline faculty to ensure compliance with accreditation standards. It is further assumed that institutions approving learning objects for inclusion in accredited coursework will have assured quality standards in curriculum development, appropriateness of delivery modality, faculty support, and assessment of efficacy of learning objects. Guidelines based on those promulgated by the Western Cooperative for Educational Telecommunications and endorsed by the Southern Regional Electronic Campus will be used as a basis for ensuring quality compliance in all learning objects submitted and reviewed for inclusion in the repository.

As part of this proposal, a lead team of FSU ODDL and FAMU will We will assess the effectiveness of technologies, individually and collectively, intrinsically and how they are used, and use the results to continuously improve the essential goal – computer science education for the workforce of the next millenium. Our underlying principle is to provide a flexible learning environment supporting multiple learning styles and allowing dynamic choices to be made by students, faculty, and programs. This assessment theme is very similar to some classical experimental investigations in computer science, for example, in operating systems, where specific algorithms for process management need to be evaluated for effectiveness in the context of real use by real humans. The assessment team will be led by FSU ODDL and FAMU and cover both testbeds.

Research has consistently found little significant difference in learning achievement among various distance learning environments or between distance learning environments and classroom environments [22,90,96]. Further self-selection by students according to personal learning style needs to be recognized as an important variable. Thus we will assess taking specifically into account the learning style of the students. Our quantitative assessment will be outcomes-based, with three classes of outcomes: success, efficiency, and satisfaction.

· Success outcomes include learning outcomes, graduation rate, and employment rate.

· Satisfaction outcomes include all relevant populations: students (while in a class, after class completion, at program graduation, after x years of postgraduate employment), employers, faculty. We measure satisfaction with learning as well as technology acceptance and usability.

· Efficiency outcomes include time invested (by students, faculty, and support team per student credit hour), re-usability of courseware (across institutions as well as over time), and costs of maintenance of technology and courseware.

In two Syracuse Ph.D. theses, Lee and Sen [63,85] have explored the technology needed to track student progress through online material. The capability to monitor and datamine such information is likely to improve as this critical for commercial portals. We will include such assessment techniques in our project as they become useful in practice.

We will supplement the strategies above with a more qualitative assessment thrust, which includes:

· External peer review: ODDL is already establishing an external refereeing process for its courses and an external peer assessment process. A similar process, using faculty from peer departments in peer institutions not will be developed for review of modules submitted for the repository. It associated with this project. (This is in addition to, and independent of, the already existing External Advisory Board that has been used to inform ODDL and Computer Science during the setting up of the distance computer science programs for Florida community colleges.) We will expand this process towill include both testbeds and the broader national community as represented by EOT (Education Outreach and Training) effort of the NSF PACI program and the NSF CILT Learning and Intelligent Systems center [16].

· Customer feedback: Using interviews and focus groups from students, faculty, academic programs, and industry to assess customer satisfaction and identify areas for change and improvement.

All of the assessment results will be used in a feedback-improvement loop to continuously improve both the technology and the courseware during and after the project. The availability of useful assessment information and its use for self-improvement, particularly on time scales shorter than a semester, is largely unavailable to standard classroom instruction. Continuous (short and long time scale) self-improvement and opening the process to all possible learning styles simultaneously are two ways in which the new systems can result in better performance over classical systems.

4: Distance Education Technology and Computer Science ResearchWe will build education specific portals as a set of special services on top of this framework. These must support the special collaborative needs of education and special services such as assessment, performance (grading) support, annotation. There are also distinctive “educational objects” – quizzes, homework, glossaries as well as the curriculum pages with appropriate hierarchical structure [18]. These will need special XML support and here we will adopt local standards as necessary and evolve these as international community efforts (such as IMS [26] and the IEEE Learning Technology Standards Committee [25]) mature. We will of course pay attention to support for key capabilities such as displaying mathematics on the Web [21] and standards for graphics (Java3D, VML, X3D etc.). This distributed object based distributed system will be designed to support curriculum material built in any web authoring system and specified either statically or dynamically (from a database). This simple statement is not easy to satisfy, as it requires unification of services such as those for customization, collaboration and events. This is a key research area as such unified services are essential for the basic strategy of allowing components from multiple academic and commercial sources. A simpler version of this challenge is well-defined XML interfaces to allow interoperability of data streams.

We expect commercial portal technology to support user customization of the environment and we have already indicated that the base service (event logging) is expected to be useful both in assessment and individualization of the learning environment. This includes two types of capabilities. Firstly the capability, probably XML based, to pick and use the components shown on a particular web-page (portal). We have designed a simple “portalML” to describe layout and source of page components and further their collaborative structure [20]. We expect this XML syntax to be a reasonable start but that we will switch to community standards as they become accepted. More interesting than this powerful but straightforward XML specification of dynamic pages, is the methodology for tracking user interactions with the user environment. As discussed in the Syracuse theses of Lee and Sen [30,43], this can be done server side when it reduces to the classic analysis of Web Server accesses logs. More interesting is the tracking of client side events where the challenge is basically datamining user relevant information. We will on one hand build in support for this as part of our event service and research extensions of the simple analyses in the two theses to automatically derive user profile and learning assessment information. This client side event information can be used to support universal access as described by Fox and Gilman from the Wisconsin Trace center [19].

Our web-based virtual university approach implies that collaboration is a service that shares web-based distributed objects [41]. Previous systems have tended to support either synchronous or asynchronous collaboration modes but based on our current experience, we will unify them for this proposal. Initial synchronous deliveries have has some success using systems like Microsoft NetMeeting, NCSA’s Habanero [23] and Syracuse’s TangoInteractive [47]. However the new requirements imply we will not use these but rather build collaboration on the event service of our base (Ninja or equivalent) framework. We will allow this to support either synchronous delivery or event archiving and later delivery of a session. Session control will be implemented in XML using the generalized portalML described above [20]. We have found that developing shared animations (for education) is too difficult in current systems like TangoInteractive, which only easily support complex collaboration-aware applications. We will use VNC [51] or equivalent technology to allow both shared display and collaboration-unaware applications, which are less flexible but much easier to author. One important research issue will be the techniques needed to provide this unified approach to collaboration.

4 Technology and Standards for Learning Environments

4.1 Overall Framework

Our approach to courseware and tools is built in terms of distributed object technology and is consistent with the collaborative university model of fig. 2, and the curriculum design model of sec. 3.1. Many commercial and academic projects developing the key technology ideas are primarily driven by areas like e-commerce and commodity Web resources, but only later and after appropriate customization can these be applied to education. We will build on the emerging integration of distributed, component, and Web technology with our approach being compatible with the many competing candidates for the base infrastructure. These include CORBA, Jini, Enterprise Javabeans, Web-linked databases, and a variety of XML and Java based systems such as SOAP [89] from Microsoft and iPlanet [57] from Sun. We consider Ninja [75] from UC Berkeley and E-Speak [30] from Hewlett Packard as interesting new approaches, and we will evaluate the new release of Ninja over the summer as a possible infrastructure for this project. We also see some analogies between the requirements for a learning environment and the successful but controversial Gnutella [46] or Napster [73] type distributed archive technology for multimedia material.

To ensure that we can protect our investment we will adopt well-defined interfaces implemented in terms of XML and if necessary change our implementation as technology evolves. We introduce a 3-tier architecture with client, server and backend resource and the two interfaces, as shown in Fig. 4 [39]. This approach has been adapted successfully in the Gateway Web based computing project [4] with the use of two interfaces separating the user and system object view and insulating both the user interface and repository resources from the changing server infrastructure. As a simple example from the relational database field, resourceML would define the table structure used to classify the data while portalML would support user queries in SQL. Our application is detailed later in fig. 5 and the backend includes the courseware as well as the events (information nuggets) describing the users and their interactive sessions. Our proposed system will support the courseware developer who is adding or editing modules as well as the learners and teachers accessing the courseware repository. In addition it will provide tools to support person to person and person to database interactions. As discussed in Sec.4.2, existing standards efforts have provided a good start to these interfaces although they base on a less sophisticated client server model and essentially merge these two interfaces. In Sec. 4.3, we elaborate our technical approach built around the concept of a collaborative portal.

4.2 Standards and Learning Objects

A number of efforts to develop standards have relevance to our proposed research. We will focus on two very recently published efforts, which define standard properties of learning objects. One standard is the Instructional Management Systems (IMS) Learning Object Metadata (LOM)[56], which is based on the IEEE Learning Technology Standards Committee (LTSC) Learning Object Meta-database [54]. The second standard, a Sharable Courseware Object Reference Model (SCORM)[3], was developed in collaboration with IMS and IEEE LTSC by the Advanced Distributed Learning Initiative (ADL) for the US Department of Defense. Both standardization efforts have built upon previous efforts, resulting in current standards that support a richer set of educational resources than their predecessors. Furthermore several other general standardization efforts, such as the Resources Description Framework Model and Syntax Specification [62] and the Synchronized Multimedia Integration Language [53], from the World Wide Web Consortium are not directed specifically at educational materials but will be important for our project.

IEEE LTSC [54] defines learning object metadata including the specification of properties such as technical and educational properties (such as format and interactivity), meta-metadata, (ownership) rights, relationships (between objects), annotation and classification. IMS [55] has built on this basic metadata, Enterprise properties (such as personal data for students) and a general framework for content re-usability[56].

The ADL SCORM standard is intended to produce "web-available, sharable courseware objects that are reusable in the development of technology-based instruction, portable across different platforms, accessible through the use of meta-data standards for identifying and locating them, and durable across different versions of operating systems, browsers, and other supporting software[3]". The ADL Initiative hopes to provide a starting point for the next generation of advanced learning technologies that can be highly adaptive to student needs. The resulting specifications include a Course Structure Format (CSF), that is an XML-based representation of a course that can be used to describe all course elements, structure and external references necessary to move a course from one learning management system (LMS) to another. Also, they specify a run time environment that includes the specific launch protocol to initiate web-based content, a common content-to-LMS application program interface, and a data model defining the data, which can be exchanged between a learning management system and executable content at run-time. The standard includes metadata for describing the course content, content metadata (which incorporates the IMS Learning Object Metadata core elements) and raw media metadata. Central to SCORM is the concept that courses can be broken up into blocks (collections), objectives, and assignable units (au) that could be combined under the control of an intelligent learning management system. Course completion requirements and pre-requisites are included. The concept of a collection and the flexible assembly of other collections and au’s into a new collection is clearly important for building courses from re-useable modules. Within SCORM, assignable units are key building blocks in the overall scheme to track a student's progression through a course. The assignable units contain content and implement the application interface that provides the student progression information needed to customize a learning management system's responses to individual students.

In our approach, the application interface is encompassed in the client side “user view” interface as shown in Fig. 4. We intend to support more collaborative flexible learning models than just computer based tutoring on which SCORM tends to focus. Further IEEE LTSC, IMS and SCORM need to be tested in commercial systems such as Blackboard[10], LearningSpace[65] and WebCT[101]. Under the leadership of co-PI Thompson a working group organized by SURA will explicitly examine the exchange of learning objects between these three commercial systems and a compatible XML resource definition will be basis of this.

Thus we see that LTSC, IMS and SCORM provide useful starting points for our project, which is consistent with the curriculum design model of sec. 3.1. We do expect to need to make major extensions in several areas and we will work with the community to ensure that lessons learnt from our project are integrated into the standards activities.

4.3 Collaborative Portals

It is unrealistic today for any one to build a complete online education environment from scratch: rather one must integrate a system from a variety of different sources. This motivates the standards for re-usable objects described in the previous section.

In this proposal we take an approach that in modern parlance is called an educational portal. A portal employs a modern distributed object framework (as discussed in Sec. 4.1; we will evaluate Ninja for this) and uses it to support distributed learning objects and services with the two interfaces defined above. We bring in substantial experience in this approach for both computing and education applications and are developing an integrated approach with the NCSA Alliance. We adopt a layered approach with one set of capabilities common to all portals and then specialize to different applications. Here we view a portal as “just” a web interface to a particular application area. [39]

The general properties of any portal include storing, accessing and searching for distributed objects (which of course include web pages) in a repository. Further we have general services such as security and collaboration where the latter is particularly important for education as it enables the synchronous or asynchronous interactions between students and teachers. Further general portal capabilities include layout (of the rendered objects on a page), provision of metadata, universal access, user customization and performance (through use of mirror or proxy servers). We will research the use of the client-server interface (see Fig. 4) to define the object properties of relevance to these functions and as usual express them in terms of XML as “portalML” [38]. As shown in the SCORM standard, one must support both base educational objects (modules) and their integration into lectures, courses, curriculum etc. We did this with our early WebWisdom system [34, 35] and an attractive interface for this can be seen in commercial software such as RealJukebox [81], which is designed to collect multimedia objects, which are simpler but have interesting points in common with learning objects. This software also supports neat layout customization through different “skins”.

Returning to education, one must support special services such as assessment, performance (grading) support, and annotation. There are also distinctive “educational objects” – quizzes, homework, glossaries as well as the curriculum pages with appropriate hierarchical structure. Here we will extend SCORM and IMS but separate the “user view” from the basic resource specification. The latter (“system view”) describes the learning modules stored in the shareable courseware repository (see Fig. 2 and Sec. 3.) We will of course pay attention to support for key capabilities such as displaying mathematics and other symbolic notations on the Web [40] as well as standards for graphics (Java3D, VML, X3D etc.). This distributed object based system will have to support curriculum material built in any web authoring system and specified either statically or dynamically (from a database). This simple request turns into a serious challenge, as it requires the unification of services such as those for customization, collaboration, and events. This is a key research area as such unified services are essential for the basic strategy of allowing components from multiple academic and commercial sources. A simpler version of this challenge is well-defined XML interfaces to allow interoperability of data streams.

This appears a complex daunting agenda but fortunately many of the capabilities are provided by the new generation of Internet infrastructure. Therefore for this proposal we can focus on a few key issues. We will assume that new browsers (Internet Explorer 5 and Netscape 6) will have satisfactory support for the W3C document object model [101] and XML. This already provides a nice way of specifying collections that is consistent with SCORM. We will build some simple layout tools supporting a portalML [38] allowing natural grid and flow layouts (using a Java AWT notation). We assert that that key new capability shown in fig. 5 is an event service that allows one to receive and send time-stamped tagged messages. These events define the state of each portal page and can be used to support user customization by saving the event queue. The event queue is designed as a distributed (XML) database to support guarantees of robust delivery and performance through replication of shared events. The event log can also be used in assessment of both the student and the learning material as it records the user’s interactions with the environment. As discussed in the Syracuse theses of Lee and Sen [63,85], this can be done server side when it reduces to the classic analysis of Web Server accesses logs. More interesting is the tracking of client side events where the challenge is basically datamining user relevant information. We will on one hand build in support for this as part of our event service and research extensions of the simple analyses in the two theses to automatically derive user profile and learning assessment information. This client side event information can be used to support universal access as described by Fox and Gilman from the Wisconsin Trace center [37].

Our web-based virtual university approach implies that collaboration is a service that provides the sharing of web-based distributed objects [82]. Previous systems have tended to support either synchronous or asynchronous collaboration modes, but based on our current experience we will unify them for this proposal. Initial synchronous deliveries have had some success using systems like Microsoft NetMeeting, NCSA’s Habanero [47], and Syracuse’s TangoInteractive [93]. However the new requirements imply we will build collaboration in terms of the event service of our base (Ninja or equivalent) framework. We will allow this to support either synchronous delivery or event archiving and later delivery of a session. Session control will be implemented in XML using the generalized portalML described above [38]. We have found that developing shared animations (for education) is too difficult in current systems like TangoInteractive, which only support complex collaboration-aware applications without difficulties. We will use VNC [97] or an equivalent technology to allow both shared display and collaboration-unaware applications, which are less flexible but much easier to author. One important issue of our research will be the techniques needed to provide this unified approach to collaboration.

We are already building examples of this architecture shown in fig. 5, with an event service, which is designed to support the performance of immediate forwarding of object state changes that is needed by synchronous collaboration. This is combined with the archiving of events to support later asynchronous browsing of the course by users accessing the persistent database. We ran in difficulties with TangoInteractive due to its extensive use of browser-based software. In this approach we will avoid putting significant client side logic into a browser but rather use a “personal server”. Here we view the browser (on a PC or hand-held device) as one particular rendering device – it contains the code to support rendering but the session logic and important data is controlled client side by a server. This approach is consistent with systems like Ninja and allows a single user session logic to support multiple display devices including cross disability access such as a pure audio rendering for the visually impaired.

One continual area of challenge is the variable quality in digital audio and video conferencing. Higher speed in networking and improving quality of service will address some of the difficulties. We will track the ANL/NCSA Access Grid project [5] at the high end, but for many educational uses commercial systems like RealAudio/Video can be used. In our multi-paradigm framework, we will allow the user to switch dynamically between interactive audio-video technology and the more reliable non real time systems (like RealAudio) whose larger buffer sizes are less sensitive to the lack of quality of service on today’s internet. We have noted in our classes between JSU and Syracuse that we could use the more robust approach when the teacher is lecturing and interacting with the class through the chat rooms rather than the audio channel. This accounts for well over 95% of the time of a typical lecture.

We will use our Gateway computing portal to build a generic portal supporting portalML and resourceML which will be operational over this summer. This will include a prototype event and layout service and we will use experience from this in evaluating the possible new object web infrastructures discussed in Sec 4.1. We also expect completion of planning for the SURA effort to build an interoperable framework for key commercial systems. This should put us in a good situation at the start of this project to add sophisticated capabilities based on the IMS and SCORM standards needed to support a prototype of the courseware repository. During the initial 6 months of the project we will make simple choices for collaborative services -- perhaps using TangoInteractive or the Access Grid combined with a simple "shared browser". In the spring of 2001, we expect to add the key collaborative capabilities based on the event service so that we can start using this research system in our courses starting in the second year of the project. We will expand and evolve his research effort in directions suggested by our experience with the collaborative university.

5 Research and Management and Outreach Plan

5.1: Management Plan and Budget

The principal investigator has substantial experience with running large multi-institutional projects funded by NSF and DARPA as both project PI and co-PI. For a project of this size, we intend a steering committee containing leaders of technical activities and site representatives. This will discuss and approve major decisions. We will establish an oversight group, which will review general approach and supply vision and connectivity to national scene. This will help in the qualitative assessment plan of Sec. 3. Initially we intend to work with the NSF PACI EOT to provide the members of and suggestions for the outside review panel. The operation of the project will have a critical input from a "user group" of faculty and students which will be initially led by Jackson State University and allow direct input from the involved faculty and students.

The proposed budget is about $900K per year for five years. We see that the need to iteratively develop and assess new curriculum as well as the technology to deliver it, requires the relatively long five-year duration. The budget is split into activities as follows: Technology and Standards $275K, Assessment $125K, Management and meetings $35K and the remainder to courseware development and academic and technical network building.

5.2 Research Plan

We divide the activities of our project into four broad areas:

a) Infrastructure; administration, workshops, training and facilities.

b) Curriculum development and delivery; assessment (Sections 2 and 3.3)

c) Technology evaluation, research, standards (Sections 3.1, 3.2 and 4)

d) Deployment and support of courseware Repository and delivery systems

The project will hold two major working meetings each year. and the The first one, to be held about 3 months after the start date, will settle on the detailed implementation plans. In the first year we will set up the three groups described in sec. 5.1; the a steering committee, a user group and

an external review board. We will ensure during this first year that each HBCU has the necessary distance education infrastructure (computer labs and network connectivity) and staff needed to provide the instructional technology support (area d) above). We will start the faculty and staff training at the end of the first year and continue on this in an ongoing fashion. We will develop and offer prototype classes during the first year but the major initial effort will be a curriculum review in the HBCU network. This will define computer and computational science focus areas such as software engineering, numerical methods, operating systems etc.

We will evaluate and compare the curricula with respect to both the IEEE/ACM Curricula 2001 recommendations and the CSAB/ABET Criteria. As described in sec. 3.1, the curricula will analyzed in terms of the student acquisition of skills needed by potential employers such as business, industry, and government. We will analyze the currency and relevancy of the curricula and finally identify strengths and weaknesses of the curricula in the HBCU network. This will be compared with a corresponding analysis of the Florida community college testbed. This will determine which courses are candidates for collaborative development. We expect to find collective strengths and weaknesses as well particular departments having special needs or capabilities. We will then develop courses for which a need has been identified and which fit well with distance education delivery.

In the first year, the technology group (area c)) will first identify appropriate initial approaches from existing commercial and academic distance education systems. These will be used in the initial HBCU network delivery. We will combine the HBCU and Florida needs analysis with an object web technology evaluation to provide the detailed plan for the collaborative portal research described in sec 4.3. This new approach will start to be used in year 2, and be extensively deployed in year 3 and evaluated and be refined in years 4 and 5.

In year 2, theFollowing the intial year's curriculum review, in year 2, the HBCU network will focus on course development and delivery. The assessment process of sec. 3.2 will provide feedback to course developers, deliverers and the technology group. This iterative feedback will drive the project. Here we expect to start dissemination in a major fashion.

Years 3 – 5 will be Iterations of Years 1 and 2, with thebut will add additional testbed schools (from sources described in Sec. 2.1) and courses. Year 5 will be aimed at capturing all the lessons and organizing our results so they can drive further such efforts. This will be a valid time to gauge the degree of success for the overall project.

5.3 Dissemination of Results

Dissemination of the results of this endeavor are two dimensional. In the first dimension, the reusable learning objects called modules contained in the repository, will be available on the web for use by universities participating in this project as well as universities who learn of the existence of these modules through research publications and presentations. The second dimension includes the publication and presentation of the research including but not limited to the success and failures of specific modules and findings on the resources and portal research and applications. Conferences targeted for publications include ADMI (Association of Computer and Information Science and Engineering Departments at Minority Institutions), MU-SPIN [72], EDUCAUSE, Journal of Small Colleges [59], and the ACM Special Interest Group on Computer Science Education [2]. As described in detail in sec. 6.5, we will take advantage of the many contacts of the NSF PACI (NCSA and UCSD partnerships) EOT (Education Outreach and Training) for further outreach and dissemination. Further as described in the International section, we have in place contacts to ensure an initial exploration of ideas for collaborative university partnerships outside the USA. This has new technical and institutional challenges.

6: Capabilities of the Participating Institutions and Results from Previous NSF Awards

6.1 Florida State University

The principal investigator Geoffrey Fox has moved from Syracuse University (CSITNPAC) to the Department of Computer Science and new School of Computational Science and Information Technology(CSIT) at Florida State and brings substantial experience in both collaboration technology and novel computer science (Internetics) curriculum [32,33]. This was developed and delivered with Nancy McCracken at Syracuse, Jackson State and other participants. as a collaboration between Syracuse, JSU and MSU.

FSU is also represented by the ODDL, which supports distance learning as described in sec 2.2 and the International appendix. Our project will leverage ODDL’s existing assessment unit as well as exchanging technology and course modules. ODDL and CSIT combined with a rapid expansion of the FSU computer science department reflect the commitment of FSU to the teaching of Information Technology and its use in all aspects of research and education. Note that in 1999, there were 55 courses offered on-line at FSU to a total of 1800 students; this statistic is increasing rapidly and excludes “simple web-enhanced” courses

6.1.1 NSF Grant: Center for Research in Parallel Computation

co-PI Geoffrey Fox (while at Syracuse) CCR-9120008, $997,564 in period 2/1/97 - 4/30/00.

This Science and Technology grant was led by Ken Kennedy at Rice University and involved research in parallel computing (most recently for Fox concentrating on Java as in [14]) and of particular relevance to this project, several HPCC education activities. Most recently this involves a co-authored book where Fox is coordinating the applications sections. CRPC pioneered a set of collaboratively developed HPC courses at the (then) supercomputer centers where Fox developed several modules. These developed the early internetics ideas [32,33] and prototypes of education technology later used in DoD work [35,36]. Fox’s work on computing and education portals in the NCSA Alliance (see Sec. 6.5) is also core to this proposal.

6.2: FAMU

Florida Agricultural & Mechanical University, founded in 1887, is an HBCU land-grant institution, which educates approximately 12,000 minority students each year. The Computer and Information Science department has a 94% minority population of approximately 700 undergraduates and 50 graduate students. The department brings expertise in assessment and the use and evaluation of Internet courses. The faculty, Dr. Sara Stoecklin and Dr. Marion Harmon, have been actively involved in the development and review of curriculum and courses at FAMU and other universities during the last 15 years. They have served on university curriculum committees at various levels and on curriculum development boards at universities and industry.

6.2.1: FAMU NSF Grant: Software Engineering Research Education Laboratory (SEREL)

PI: Dr. Sara Stoecklin Renewed Support: (from previous funding)

Award Number: EIA-9906590 1999-2004 – $2,500,000

Publications: new grant (approximately 15 from previous activity)

This Florida A&M University (FAMU) Minority Institution Infrastructure proposal was centered on the enhancement of a major computing facility located within the Department of Computer and Information Science (CIS). While the grant has only been in existence for one/half of an academic year, the results are impressive. The publications (15 thus far for this new grant), presentations, research projects, research activities, and previous funding successes are fully documented on the web at the address http://www.cis.famu.edu/~iimi. . Additionally, FAMU participated in a CREST grant entitled “Center for Distributed Computing: Theory, Application and Practice”. This grant, HRD – 97070076 1997-2003 for 5,000,000 dollars has been renewed for the past three years and has 75 publications. The mission of this grant was to develop the infrastructure and inter-disciplinary cooperation that will increase the number of minority students enrolling in and successfully completing masters and Ph.D. degrees in computer science. Successful results are documented at http://www.cis.famu.edu/~crest.

,

6.3 Jackson State University

Jackson State University (JSU), is the urban university of Mississippi and enrolls approximately 6,500 students. The primary goal of the School of Science and Technology, and the new School of Engineering, is to develop top quality scientists and engineers who can advance knowledge and address the technical problems facing the nation and the world. Particularly relevant to this proposal, JSU has graduated more African Americans in Computer Science than any other university in the United States. Among African Americans in Mississippi Institutions of Higher Learning, JSU has enrolled 53% of all Chemistry majors, 54% of all Biology majors, 66% of all Computer Science majors, 69% of all Mathematics majors, and 80% of all Physics and Atmospheric Sciences majors. Thus, JSU will continue to provide significant numbers of technical graduates for the current and future workforce.

6.3.1: JSU NSF Grant: Connection to the Internet

PI: W. Brown. Grant from the Division of Advanced Network Infrastructure and Research (Network Infrastructure Program)

Award 9985957 was made on 03/21/00 for $ 309,038.00 for 23 months

This new award indicates JSU’s readiness to lead the HBCU Collaborative University with a state of the art network connection.

Florida Agricultural & Mechanical University, founded in 1887, is an HBCU land-grant institution, which educates approximately 12,000 minority students each year. The Computer and Information Science department has a 94% minority population of approximately 600 undergraduates and 25 graduate students. It brings expertise in assessment and the use and evaluation of Internet courses.

FSU is also represented by the ODDL, which supports distance learning with the principle that the same education should be available to all FSU students, whether residential and distance. Their current model includes strong materials-based support for teacher and learner; optimal use of Internet bandwidth for communication, interactivity, and delivery; and a mentor system that provides low-ratio student support and scalability at the faculty level. This project will leverage ODDL’s existing assessment unit. ODDL and CSIT combined with a rapid expansion of the FSU computer science department reflect the commitment of FSU to the teaching of Information Technology and its use in all aspects of research and education. Note that in 1999, there were 55 courses offered on-line at FSU to a total of 1800 students; this statistic is increasing rapidly and excludes “web-enhanced” courses.

6.4 Mississippi State University

The original collaboration between Fox, Brown and Thompson was sponsored by the Programming Environment & Training (PET) effort of the DoD Major Shared Resource Centers program - led by the NSF ERC at Mississippi State. It involved regular semester undergraduate and graduate CS courses, which were later, delivered by JSU to other HBCUs – the prototype of our proposed HBCU college network.

As a part of its commitment to an NSF Engineering Research Center (ERC), Mississippi State created a new cross-disciplinary graduate program in Computational Engineering in 1991. Computational engineering is an interdisciplinary program across engineering, computer science, and mathematics managed by the College of Engineering and the faculty of the ERC. A goal of the program is cross-disciplinary education that must include study of a computational engineering technology area, numerical mathematics, and high performance computing. A student may earn the M.S. or Ph.D. degrees. Entry into this graduate program can be with a BS degree in any physical or biological science, or in engineering or mathematics.

The ERC has also used its research program to enhance undergraduate education at Mississippi State by seriously involving undergraduates in research projects at the Center throughout the academic year, as well as operating a summer REU program for students from other universities and colleges. Since 1990, the ERC has awarded assistantship or wage stipends to approximately 870 students to be involved in the research of the Center, with about half being undergraduate students and half being graduate students. Almost all of these students have worked with the research teams of the Center under the direction of a faculty member or a senior graduate student, while others have worked with computing services or publishing in support of the research. In addition, a number of other students have been involved in the research of the Center through special problems, independent study, and activities in courses taught by ERC faculty.

Each year since 1991, the ERC has offered a summer internship program supported by funding from the NSF Research Experience for Undergraduates (REU) program. The students come for a ten-week research experience under the mentorship of one of the ERC researchers. Most summers, a few students from Mississippi University for Women and from Jackson State University are included in the program and are supported by other funds.

6.5 EOT-PACI, the Education, Outreach, and Training Partnership for Advanced

Computational Infrastructure seeks to develop human resources through the innovative use of emerging information technologies to understand and solve problems. The participants in this proposal will leverage their relationships with EOT-PACI for general national dissemination of results, increased participation of minority serving institutions, and technical cooperation on educational portals. As part of its dissemination efforts, EOT-PACI maintains a web resource that is nationally visible and used (http://www.eot.org). Roscoe Giles and the Boston University team are responsible for the content of this site and for the development of linked repositories of interest to the computational science education community. As part of this effort, the Boston University team will incorporate courseware components and resources generated by this project into the set of resources at the EOT-PACI site. EOT-PACI is working closely with EDUCAUSE on the Advanced Networking with Minority Serving Institutions (ANMSI, http://www/anms.org) project. The EOT-PACI component of this effort concentrates on making advanced network applications available to MSI participants through workshops, training, and general efforts to be sure that MSI faculty and staff are better represented in the national activities involving advanced network applications such as the Grid Forum [45] and portals [20] organizations. As soon as it is possible, we will incorporate the results of this project into the framework of activities that we offer to MSI's through the ANMSI project. This can serve as an outreach vehicle to additional HBCUs as well as Hispanic Serving Insititutions and Tribal Colleges. Allison Clark (NCSA) and R. Giles (BU) are principal contacts for the EOT-PACI ANMSI effort. The joint activities under this proposal will be coordinated through Boston University.

International Collaborations

China: International Collaborative Web University ICWU

Fox and Professor Xiaoming Li, now chair of the computer science department at Peking University, established a strong collaboration during the three years Li visited NPAC at Syracuse University. This included an early successful experiment in distance education in 1996 with a course in Internetics [33] taught from Syracuse to Harbin Institute of Technology in North China. This necessarily used asynchronous technology quite different from the later JSU experiments at NPAC. This led to a proposed extension of this as the ICWU with an initial exchange of courses between Peking, Syracuse and Bristol England (UWE). ICWU (International Collaborative Web University) could be viewed as an early vision of the concept described in fig. 2. The differences in timing, course size and student preparation clearly require the modest size learning objects proposed here to allow customization for each student body. We intend to build on this now Fox has moved to FSU with an exchange program of students and more senior researchers between FSU and Peking. The collaborative education portal described here is clearly far more suitable for cross continent education than the inflexibly synchronous TangoInteractive system. We hope to expand this fruitful collaboration (which also includes work on parallel Java) using both the curriculum and technology proposed here.

We can also note significant interest in collaboration between the European Union and USA in this area with Fox and the PACI EOT invited to a recent meeting on this subject with substantial European collaboration. The EHR division of NSF sponsored this in February at SDSC in San Diego and the discussions there were compatible with ICWU and the project proposed here. Again visiting programs and exchange of technology and curriculum should benefit both this project and our European colleagues. Cross-continent distance education obviously has many difficult and important technical, cultural and institutional issues.

Africa: "The Computer Science Curriculum and the Next Generation of Education Technologies" proposes to not only increase the minority participation in the advancement of computer science and computational science in the United States, but also among participants in developing nations throughout the world. In particular, we will identify opportunities for cooperative international activities, that form linkages with existing programs among the partnership, with developing countries.

With this goal in mine researchers from the partnership met with the Abdus Salam International Centre for Theoretical Physics (ICTP Trieste, Italy) Director Miguel Virasoro, National Centre for Mathematical Sciences (NCMS Accra, Ghana), and the University of the Western Cape (Cape Town, South Africa) International Relations Director Jan Persens during the summer of 2000. As a result a second series of meetings between the Director of ICTP and NCMS has been scheduled for June 2000 at the 2000 World Automation Congress and in July at the University of the Western Cape. At these meetings formal agreements between these institutes and members of the partnership, agreeing to work collaboratively on various projects supported in part by this proposal will be placed into effect.

FSU Overseas:

Florida State University has a real and growing presence outside North America. Their branch campus in Panama, which operated since the 1950s as a service primarily to US citizens serving in the US Canal Zone, has recently expanded into a large campus in the former US government facilities. The Computer Science program has been active there for about 10 years, and there are plans to significantly expand that program, for US students studying abroad, Panamanian citizens, and more generally as a gateway to South America. FSU also has facilities and active study abroad programs in Florence, Italy (started in 1966); London, England (started in 1971); and Torremolinos, Spain (started in 1997). The new availability of computer science at a distance is expected to impact those programs significantly. Other initiatives that are in process but not complete could result in branch campuses in India, Russia, United Arab Emirates, and Viet Nam, all of which would feature Computer Science as one of their first programs, offered using the FSU branch campus system discussed in section 2.2.1. This project would interact with FSU overseas in exchange of and synergy between technology and shared course modules. In particular the special demands of overseas students will stress test the true reusability of the modules and the appropriate granularity of their preparation. This will add value to our participation in the standards forums as it will test proposals in a broader context. We will of course test again the multi-layer collaborative university approach of fig. 2 with separated functions for preparation, teaching, and mentoring.

FACILITIES

Florida State University's School of Computational Science and Information Technology, is the intended home of this project. The School is housed on the first and fourth floors of the Dirac Science Center in approximately 11,000 square feet. This facility will provide the core office space, meeting facilities and computer network infrastructure for faculty, students and others involved in this project.

Current Research Equipment

The School of Computational Science and Information Technology (CSIT) operates a host of equipment relevant to this project as well as additional computer equipment for computational science research. These include:

· A Linux based web server (450 MHz CPU, 60 Gbytes disk, 100 Mbit/s connection to the internet).

· A dual processor (333 Mhz) Sun ES3500 file server, with 1 Gbit/s access to the network.

· A host of computers used for computational science including: 5, four R10000 processor SGI Origin 200s (180 Mhz) each with 1 Gigabyte memory and 27 Gigabytes of disk, one single 180 Mhz CPU Origin 200 (x-terminal server) one SGI maximum impact with one Gigabyte memory, and several SGI O2 workstations each with 1 Gigabyte memory.

· A 32-node Pentium Pro computing cluster with dual processors (400 Mhz), 256 Mbytes memory and 18 Gbytes disk per CPU, and a 100 Mbit/sec Ethernet. The cluster supports the activities in physics in collaboration with Jefferson Labs in Newport News, VA.

· A 16-processor, IBM SP2: 8 wide nodes, each with 1 gigabyte RAM, 12 gigabyte disk and 1024 Mbytes of memory, 8 thin nodes each with 256 Mbytes RAM and 12 gigabytes disk supporting computational chemistry and physics.

· Two alphas ES40 from Compaq with 4 CPUs and 8 Gbytes RAM each for theoretical chemistry research.

· One IBM RS600-590 used as a backup file server system.

· Approximately 80 desktop workstations or PC's

· Multimedia recording facilities for creating CD-ROM’s and laser disks.

· A bank of 24 modems (56 Kbaud) for use at home or by those on travel.

· A Visualization Laboratory including:

o 1 infinite Reality Onyx with 2 pipes, 4 R12000 processors, 250 Mhz processor, 8 Mbyte cache, 2 Gbyte RAM, 128 Mbyte texture memory, and 200 Gbyte disk farm. Eight of the 10 disks are striped pairwise for faster I/O.

o 1 rear projection 8'x16' PowerWall, capable of stereographics display. It resides in the seminar room and is used together with two 24" monitors for visual research, classroom activities, and presentations.

o A wide variety of Silicon Graphics computer systems and workstations to support graphics development.

Currently CSIT is evaluating responses to an RFP for a $8M high performance computer funded by the State of Florida to support their research. This important facility will be available to support some advanced computational science classes. Further Fox is establishing his research group at FSU and this includes some 20 Sun and PC servers set-up to run Oracle and allow student uses in classes for Internet and parallel computing topics. These machines (in the similar configuration at Syracuse) were routinely used by distance education students.

The campus network provides access to a number of regional, national, and worldwide networks including ESnet, NSFnet, HEPnet, BITnet, FIRN and SURAnet. In addition, two T1 connections to Esnet, via the University of Texas at Austin and Oak Ridge National Labs, are currently in use. The FSU campus backbone is a 1 Gigabit FDDI ring that connects the individual research groups involved in this effort. Florida State University is a member of Internet 2.

FSU’s Academic Computing & Network Services (ACNS) is acquiring, installing and integrating the computer systems (file, mail, web, news, security, and database servers) and software (CourseInfo Enterprise Edition and Oracle) necessary for the long-term delivery of these distance-learning courses. Currently the system includes 38 servers (25 Suns, 11 IBM's and 2 SGI's) with over 300 Gbytes of disks storage for student and faculty use. This system will eventually be used by all of FSU for delivery of web-enhanced courses. In addition they are providing phone and online support for users of this system. Through agreements with various vendors they are able to distribute standard software, such as browsers, ftp and terminal emulation programs, etc. on CD-ROM's to all FSU students. ACNS also provides off-campus connectivity to the Internet for students and faculty via approximately eight hundred 28.8k or 56k modems connected to the appropriate rotary dial-up facilities.

References

1) ACM Curricula Recommendations. http://www.acm.org/education/curricula.html

2) American Computing Machinery Special Interest Group on Computer Science Education (SIGCSE). Proceedings of the Twenty-Seventh SIGCSE Technical Symposium on Computer Science Education Philadelphia, Pennsylvania February, 1996.

3) ADL SCORM: Sharable Courseware Object Reference Model, http://www.adlnet.org/ADL-TWG/documents.htm

4) Erol Akarsu, Geoffrey Fox, Tomasz Haupt, Alexey, Kalinichenko, Kang-Seok Kim, Praveen Sheethaalnath, and Choon-Han Youn, Using Gateway System to Provide a Desktop Access to High Performance Computational Resources, Proceedings of HPDC-8 Conference, Redondo Beach Ca., Aug 3-6, 1999, IEEE Press. http://www.osc.edu/~kenf/theGateway/ and http://www.npac.syr.edu/users/haupt/WebFlow/

5) Argonne National Laboratory NCSA Alliance Access Grid Digital Conferencing System,
http://www-fp.mcs.anl.gov/fl/accessgrid/default.htm

6) Barry, M. and Runyan, G., A Review of Distance-Learning Studies in the U.S. Military, The American Journal of Distance Education. Vol. 9, No. 3, pp. 37-47. 1995.

7) Bereiter, C. (1991, April), Implications of connectionism for thinking about rules. Educational Researcher, 20, 10-16

8) Bernholdt D.E., Fox G.C., Malluhi Q., Markowski R., McCracken N., Mitra D., Podgorny M., Scavo T., Synchronous Learning at a Distance: Experiences with Tango, Proceeedings of SC98 Orlando, IEEE. http://www.npac.syr.edu/projects/training/Papers/sc98/

9) Bernholdt D.E., Fox G.C., McCracken N., Podgorny M., Network-Based Remote Collaboration and Training: Progress and Plans, Internal technical report http://www.npac.syr.edu/Projects/DoD/white-paper-collab-train-990510.html

10) Blackboard Inc is a major provider of education technologies. http://www.blackboard.com/

11) Bornstein, R.F. (1988). Radical behaviorism, internal states, and the science of psychology: A reply to Skinner. American Psychologist, 43, 819-821

12) Bruner, J.S. (1973). Some elements of discovery. In J.S. Bruner, The relevance of education. New York: Norton.

13) Bruner, J.S., Goodnow, J. & Austin, G.A. (1956). A study of thinking. New York: Wiley.

14) Bryan Carpenter and Geoffrey Fox and Sung Hoon Ko and Sang Lim, Object Serialization for Marshalling Data in a Java Interface to MPI, ACM 1999 Java Grande Conference, ACM Press 1999. )

15) Chacon-Duque, Fabio J., Building Academic Quality in Distance Higher Education. University Park, PA: The Center for the Study of Higher Education, The Pennsylvania State University, Fall. 1985

16) CILT. The NSF Center for Innovative Learning Technologies, http://www.cilt.org

17) Cheng, H-C, Lehman, J., and Armstrong, P., Comparison of Performance and Attitude in Traditional and Computer Conference Classes, The American Journal of Distance Education. Vol. 5, No. 3, pp. 51-64. 1991.

18) Clark, B., Comparison of Achievement of Students in On-campus Classroom Instruction versus Satellite Teleconference Instruction. Paper presented at the National Conference on Teaching Public Administration, Charlottesville, VA, March, 1989.

19) Commission on Professionals in Science and Technology: Professional Women and Minorities -- A Total Human Resource Data Compendium, May '97, published by CPST http://www.cpst.org/

20) NCSA Alliance / Java Grande Computing Portals Forum http://www.computingportals.org.

21) Dede, C. (1990). The evolution of distance learning: Technology-mediated interactive learning. Journal of Research on Computing in Education, 22, 247-264.

22) DETC, Accredited Distance Study Degrees: Graduates and Employers Evaluate Their Worth, Distance Education & Training Council, 1994.

23) Dick, W. and Carey, L. (1996). The systematic design of instruction. New York: HarperCollins Publishers.

24) Digitalthink.com is a typical commercial computer science training site: http://www.digitalthink.com

25) Dodd, J., The Credibility of Distance Education. Walton, England: Distance Education Group, The Open University. 1981.

26) Driscoll, M. (1994). Psychology of learning for instruction. Needham Heights, MA: Allyn & Bacon.

27) Edmondson, K. M. (1993). Concept mapping for the development of medical curricula. Paper presented at the Annual Conference of the American Educational Research Association, Atlanta, GA. (Eric Document Reproduction Services No. ED 360 322)

28) Educational Object Economy http://www.eoe.org

29) Ericsson, K. A. and Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist 49, 725-747.

30) E-Speak project at Hewlett-Packard, http://www.e-speak.hp.com/

31) Figueroa, L., Understanding Students’ Approaches to Learning in University Traditional and Distance Education Courses. Ann Yakimovicz, ed. Distance Learning Research Conference Proceedings, San Antonio, TX . College Station, TX: Texas A&M University. April ,1994.

32) Fox G.C., Internetics: Technologies, Applications and Academic Fields Invited Chapter in Book :Feynman and Computation", edited by A.J.G. Hey, Perseus Books (1999), http://www.npac.syr.edu/users/gcf/internetics/)

33) Fox G.C., From Computational Science to Internetics: Integration of Science with Computer Science, Chapter in a book dedicated to John Rice of Purdue (to be published). http://www.npac.syr.edu/users/gcf/internetics2/

34) Fox G.C., Internal memo on Collaborative Educational Objects, http://www.npac.syr.edu/users/gcf/webwisdomrefs/npacwwdocs.html

35) WebWisdom: Simple Perl System developed by Fox supporting dynamic collections of foils and web pages http://www.npac.syr.edu/users/gcf/byarea0.html

36) Fox,G.C. Architecture of Web-Based Training and Education System, SC97 CDROM and http://www.supercomp.org/sc97/proceedings/EDU/FOX/INDEX.HTM

37) Fox G.C., Initial discussion of use of XML for Universal Access, Shared Places on the Web: XML for Web-based collaboration and Distance Education, XML Developers Conference http://www.gca.org/conf/xmldev99/ Montreal August 19-20 1999, http://www.npac.syr.edu/users/gcf/montrealxmlaug99

38) Fox G.C., Initial Description of CPW (Collaborative Portal on the Web) - http://www.npac.syr.edu/users/gcf/offbeatxmloct99/

39) Fox, G.C. Portals and Frameworks for Web Based Education and Computational Science, Proceedings of the Second International Conference on the Practical Application of Java, Editor Omer Rana, Manchester England April 12-14 2000 http://www.practical-applications.co.uk/PAJAVA2000/index.html and http://www.new-npac.org/users/fox/documents/pajavaapril00/

40) FSU ODDL Workshop on Symbolic Notations on the Web, October 8, 1999, Tallahassee, http://cologne.oddl.fsu.edu/~pdragovitsch/WS/.

41) Gagne, R.M. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart & Winston.

42) Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books

43) Gehlauf, D.A., Shatz, T. and Frye, T. Faculty Perceptions of Interactive Instructional Strategies: Implications for Training. The American Journal of Distance Education. Vol. 5, No. 3, pp. 20-28. 1991.

44) Glaser, R. (1984). Education and thinking: The role of knowledge. American Psychologist, 39, 93-49.

45) Grid Forum Home Page http://www.gridforum.org

46) Gnutella: Counter Culture distributed information system http://gnutella.nerdherd.net

47) Habanero Home Page at NCSA - http://havefun.ncsa.uiuc.edu/habanero/

48) Hall, B., Ward, A.W. and Comer, C.B. Published Educational Research: an Empirical Study of Its Quality. Journal of Educational Research. Vol. 81, No. 3, pp. 182-189, 1988.

49) Hansen R.E. (1995). Five Principles for guiding curriculum development practice: The case of technological teacher education. Journal of Industrial Teacher Education, 32 (2), p. 30-50.

50) Harmin, M. (1994). Inspiring active learning: A handbook for teachers. Alexandria, VAS: Association for Supervision and Curriculum Development.

51) Heinich, R., Molenda, M, Russell, J. & Smaldino, S. (1999). Instructional media and technologies for learning (6th ed.). New Jersey: Prentice-Hall, Inc.

52) Holland, J. & Skinner, B.F. (1961). The analysis of behavior. New York: McGraw-Hill.

53) P. Hoschka, Synchronized Multimedia Integration Language (SMIL) 1.0 Specification, http://www.w3.org/TR/REC-smil.

54) IEEE Learning Technology Standards Committee (LTSC), http://ltsc.ieee.org/

55) IMS (Instructional Management System) Project from Educause, http://www.imsproject.org/

56) IMS Content Packaging Information Model, http://www.imsproject.org/content/cpinfo01.html

57) iPlanet e-commerce software from Sun Microsystems, http://www.sun.com/software/iplanet/

58) Java Academy for Middle and High School Students offered Spring 1999 by NPAC. http://www.npac.syr.edu/projects/k12javaspring99/.

59) Proceedings of the Third Annual Journal of Computing in Small Colleges (JCSC) Fairfield, Connecticut, April, 1998.

60) Jewett, F. The Human Computer Interaction Certificate Program at Renssalaer Polytechnic Institute: a Case Study in the Benefits and Costs of a Joint Industry/University Designed Program Featuring Integrated Delivery Systems. Seal Beach, CA: The Chancellor’s Office, California State University. 1997.

61) Johnson, D.W. & Johnson, R. (1975). Learning together and alone: Cooperation, competition, and individualization. Englewood Cliffs, NJ: Prentice-Hall.

62) O. Lassila and R. R. Swick, Resource Description Framework Model and Syntax Specification http://www.w3.org/TR/REC-rdf-syntax

63) Lee M., Information Architecture of Interactive and Customizable Learning Environments, Syracuse Ph.D. 2000, advisor G. Fox.

64) Lewis, L., Alexander, D. and Farris, E., Distance Education in Higher Education Institutions. NCES-98062. Washington, D.C.: U.S. Government Printing House. 1997.

65) Lotus(IBM) LearningSpace http://www.learningspace.com.

66) Macfarlane, C. and Smaldino, S. (1997). The electronic classroom at a distance. In. R. Rittenhouse and D. Spillers (eds.), Modernizing the curriculum: The electronic classroom. Springfield, MO: Charles Thomas Publisher.

67) Massy, W.F. Life on the Wired Campus: How Information Technology Will Shape Institutional Futures. In Diana G. Oblinger and Sean C. Gold, eds. The Learning Revolution: The Challenges of Information Technology in the Academy. Bolton, NY: Anker Publishing Co., 1997.

68) McClelland, J., Adult and Vocational Education: Implications on Research for Distance Delivery. St. Paul, MN: Minnesota Research and Development Center for Vocational Education, Univeristy of Minnesota. 1986.

69) MERLOT, California Virtual Campus Learning Object Libraries http://www.rh.cc.ca.us/projects/cvu/learning_objects.htm.

70) Moore, M., and Kearsley, G. (1996). Distance education: A systems view. Boston: Wadsworth Publishing Company.

71) Moore, M.G., Thompson, M.M., Quigley, B.A., Clark, C. and Goff, G.G., The Effects of Distance Learning: A Summary of Literature. University Park, PA: American Center for the Study of Distance Learning, 1990.

72) MU-SPIN Minority University Space Interdisciplinary Network Home Page http://muspin.gsfc.nasa.gov/Home1.html

73) Napster MP3 Distributed shared file system http://www.napster.com

74) National Center for Education Statistics, http://nces.ed.gov/

75) Ninja project at University of Berkeley, http://ninja.cs.berkeley.edu/

76) NPAC Distance Education Technology and Classes at Graduate and Undergraduate level, http://www.npac.syr.edu/Education/Distance/.

77) Perelman, L. (1992). School’s out: A radical new formula for the revitalization of America’s educational system. New York: Avon.

78) NSF Science Resource Studies, National Science Foundation Washington D.C. December, 1998

79) Phipps, R.A., Wellman, J.V. and Merisotis, J.P., Assuring Quality in Distance Learning: A Preliminary Review. Washington, D.C.: Council for Higher Education Accreditation. April, 1998.

80) Powell, R., Conway, C. and Ross, L., Effects of Student Predisposing Characteristics on Student Success. Journal of Distance Education. Vol. 5, No. 1, pp. 20-37. 1990.

81) RealNetworks provider of Internet Multimedia Technology and “Portals to such material” http://www.realnetworks.com

82) Roussev V., A Reference Architecture for Distributed Collaborative Applications, technical report from Univ. North Carolina Collaboration Bus Project, 1999 http://www.cs.unc.edu/~dewan/cb.html

83) Russell, T.L., The No Significant Difference Phenomenon. Chapel Hill, NC: Office of Instructional Telecommunications, North Carolina State University. 1999.

84) SCANS (1991). Secretary’s commission on achieving necessary skills. Department of Labor. http://www.ed.gov/databases/ERIC_Digests/ed389879.html

85) Sen M., Support of Assessment in a Web-based Database Environment, Syracuse Ph.D. 2000, advisor G. Fox.

86) Simonson, M., Smaldino, S., Albright, M. & Zvacek, S. (2000). Teaching and learning at a distance. New Jersey: Prentice-Hall.

87) Skinner, B.F. (1974). About behaviorism. New York: Knopf.

88) Smith, P, Dillon, C.L. and Boyce, M., A Critical Analysis of Comparative Research on Distance Learning Technologies. In Ann Yakimovicz, ed. Distance Learning Research Conference Proceedings, San Antonio, TX, College Station, TX: Texas A&M University. April 1994.

89) Simple Object Access Protocol http://www.develop.com/soap/

90) Souder, W.E., The effectiveness of traditional versus satellite delivery in three management of technology masters degree programs, Amer. J. Dist. Educ. 7(1), 1993, 37--53.

91) Suppes, P. & Morningstar, M. (1969). Computer assisted instruction. Science 166 (3903): 343-50.

92) Thomas, R. and Hooper, E., Simulations: An Opportunity We Are Missing. Journal of Research on Computing in Education. Vol. 13, No. 4, pp. 497-513. 1991.

93) Tango Interactive Collaboration System home page http://www.npac.syr.edu/tango

94) Threkeld, R. and Brzoska, K. Research in Distance Education. In Barry Willis, ed. Distance education: Strategies and Tools. Englewood Cliffs, NJ: Educational Technology Publications. 1994.

95) U.S. Bureau of Labor Statistics, Employment and Earnings, U.S. Bureau of Labor Statistics, Jan 1993

96) Valore, L., and Diehl, G.E., The Effectiveness and Acceptance of Home Study. Washington, DC: National Home Study Council. 1987.

97) VNC or Virtual Network Computing at http://www.uk.research.att.com/vnc/.

98) Vygotsky, L.S. (1981). The genesis of higher mental functions. In J.V. Wertsch (ed.), The concept of activity in Soviet psychology. Armonk, New York: Sharpe.

99) Watson, J.B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158-177.

100) W3C Document Object Model http://www.w3.org/DOM/

101) WebCT: Major commercial web-based course authoring system: http://www.webct.com/

---

* The exact composition of the meta-data will be part of the research effort as described in Sec. 4 and will incorporate standards being established by bodies such as IMS.