Journal:CyberMaster: An expert system to guide the development of cybersecurity curricula

From LIMSWiki
Jump to navigationJump to search
Full article title CyberMaster: An expert system to guide the development of cybersecurity curricula
Journal International Journal of Online and Biomedical Engineering
Author(s) Hodhod, Rania; Khan, Shamim; Wang, Shuangbao
Author affiliation(s) Columbus State University
Primary contact Email: hodhod_rania at columbusstate dot edu
Year published 2019
Volume and issue 15(3)
Page(s) 70–81
DOI 10.3991/ijoe.v15i03.9890
ISSN 2626-8493
Distribution license Creative Commons Attribution 3.0 Austria
Website https://online-journals.org/index.php/i-joe/article/view/9890
Download https://online-journals.org/index.php/i-joe/article/view/9890/5463 (PDF)

Abstract

The growing number of reported cyberattacks poses a difficult challenge to individuals, governments, and organizations. Adequate protection of information systems urgently requires a cybersecurity-educated workforce trained using a curriculum that covers the essential skills required for different cybersecurity work roles. The goal of the CyberMaster expert system is to assist inexperienced instructors with cybersecurity course design. It is an intelligent system that uses visual feedback to guide the user through the design process. Initial test executions show the promise of such a system in addressing the enormous shortage of cybersecurity experts currently available for designing courses and training programs.

Keywords: cybersecurity, expert systems, NICE Framework

Introduction

Cybersecurity has become one of the most important challenges around the world. According to a compilation published by the U.S.-based Identity Theft Resource Center (ITRC), the combination of unauthorized access to confidential information and data breaches affecting U.S. organizations and customers in 2017 resulted in 1,579 breaches, with almost 179 million records exposed.[1] This is an alarming increase, especially after considering the ITRC reported 1,093 breaches and just over 36.6 million records exposed in 2016.[2] Major types of breaches include those resulting from hacking, unauthorized access, data on the move, insider theft, accidental exposure, human error or negligence, and physical theft.[3] Recent research revealed that nearly 70% of critical infrastructure companies have reported at least one security breach during 2015 that led to the disruption of operations or the loss of confidential information. Hacking remains the leading type of cyberattack, with techniques ranging from low-tech exploits, such as phishing and social engineering, to more advanced techniques such as malware, ransomware, backdoors, exploitations, or zero-day attacks.[4][5]

The Cybersecurity National Action Plan (CNAP) is a comprehensive plan that was developed in the U.S. but can be applied worldwide to address the cybersecurity threat by taking action to expand the cybersecurity workforce, to enhance cybersecurity education and training, and to improve cybersecurity curriculum. However, several difficulties exist that hinder the spread of cybersecurity education and training, including a lack of cybersecurity skills, a lack of resources in rural areas, and a shortage of high-quality cybersecurity courses.

1. Poor cybersecurity skills: Despite the existence of cybersecurity training and personnel development programs, they are not enough as they tend to be limited in focus and lack unity of effort.[4] Additionally, there are not enough cybersecurity experts within the U.S. federal Government or private sector, and this problem is likely to be applicable to many other countries. In order to secure information systems and communication networks and maintain technical advantage over perpetrators of cybercrime, it is essential to develop a skilled, cyber-savvy workforce and an effective pipeline of future employees.[6] This requires that cybersecurity education reaches all students in the country, especially high school and college students.

2. Rural districts: Major obstacles exist to expanding the efforts to deliver high-caliber education in rural areas.[7] Rural districts in the United States make up more than half (57 percent) of all public districts in the country, while educating approximately one quarter (12 million) of all students nationwide, which make scaling and innovating across the country a challenging prospect.[8] Rural schools often face geographical isolation, shortages in specialized staff, poor physical working conditions, and limited resources[9], making access to educational programs inadequate.[7] Better technology and telecommunication can lower those barriers[7] and maximize the natural advantages of rural schools, alleviating the disadvantages.[10]

3. Course quality assurance: : Well-designed courses are those designed in a way that ensures quality course design and appropriate content while fulfilling student study requirements.[11] Evaluations of the developed courses can be difficult to quantify and may involve more than just surveys to collect meaningful data.[11] In addition, courses undergo constant revisions.[12][13] Usually, participant evaluations and/or assessment tools are often used to solicit feedback and collect data to evaluate the quality of design and course material.[14] A number of resources are required to apply these evaluation instruments, including human resources and time, which are two particularly restricting factors when it comes to rural districts. Automated evaluation of the quality of course design can be the way to address this challenge.

In an effort to inspire solutions and innovations in cybersecurity curriculum development, the U.S. National Institute of Standards and Technology (NIST) published the National Initiative for Cybersecurity Education Framework (NICE Framework). It is a partnership between government, academia, and the private sector focused on cybersecurity education, training, and workforce development. The NICE Framework consists of seven categories, 31 specialty areas, 369 KSAs (Knowledge, Skills, and Abilities), and 65 competencies. In addition, it has 444 tasks under the various specialty areas.[15]

This paper presents an interactive course design system for the rapid development of cybersecurity curriculum and training by novice instructors. It utilizes a highly visual interface and artificial intelligence techniques like rule-based inferencing to guide the design process. The system is based on a cloud-compuring platform, which offers advantages such as simplified software installation and maintenance, in addition to centralized control over versioning. Moreover, end users can access the service anytime from anywhere, sharing data and collaborating more easily.

The paper continues in the next section with a brief discussion of related work, especially the application of expert systems. The third section explains how knowledge gathering takes place in the expert system presented in this paper and how knowledge is represented using concept maps. Then, the role of a user model in providing tailored feedback for user guidance is discussed. The penultimate section discusses how evaluation of designed courses takes place, and the paper ends with concluding remarks.

Related work

The education of the cybersecurity workforce must include considerations of the security and privacy of urban and rural areas, curriculum development, and rapid development of training programs. At the same time, teachers, students, engineers, military personnel, and government employees must be trained in cybersecurity in light of the latest developments in computer networks and data communication. For example, incorporating the internet of things (IoT) architecture and security into the current curricula will empower students to gain the knowledge of how IoT can be used in a "smart cities" setting, while also allowing them to master the skills necessary to design secure IoT systems.[16] With the cybersecurity evolving fast and the level of relevant expertise lagging behind, systems based on artificial intelligence techniques can play an important role to help improve the situation.

Intelligent (smart) systems are everywhere around us, ranging from smart thermostats to smart cars. Adding intelligent capabilities to any system can reduce the workload on the human user and complement the user's efforts to achieve the desired tasks. Expert decision making systems are a widely used intelligent technique to support decision making in a specific domain. They have been successfully used— in the form of clinical decision support systems—in the medical domain to diagnose heart diseases[17][18][19], anemia[20], and diabetes.[21] Expert systems have been used in engineering for fault diagnosis[22][23], as well as in other domains such as career guidance.[24] Expert systems are usually used to address the lack of human and/or time resources. One example is an expert system for career guidance used in African high schools to address the shortage of human and time resources that the process of quality career guidance demands.[24] Other expert systems have been developed for providing academic advice to students to address the shortage of capable human advisers[25], assisting novice users in using new software [26], and helping city governments make decisions on appropriate public transport alternatives to the car.[27]

Although there are expert systems developed to help with designing courses in different areas, none of those systems targeted cybersecurity. As yet, there is no intelligent system that can make use of cybersecurity experts' knowledge and NIST's NICE Framework to guide novice instructors and trainers with the development of cybersecurity courses and study programs.

CyberMaster: An expert system to assist with cybersecurity course development

An expert system is a computer system that emulates the decision-making ability of a human expert. Expert systems can assist people with the decision making in several domains, as mentioned earlier. CyberMaster is an expert system developed to guide the development process of cybersecurity courses using the power of knowledge stored in its knowledge base. CyberMaster is composed of a knowledge base, user model, inference engine, and user interface. The following subsections present the details of each component.

Knowledge base

Expert systems use mainly if-then rules to reason about and solve complex problems.[28] The bottleneck in developing any expert system is knowledge gathering.[29] CyberMaster uses information extracted from the NICE Framework (see Fig. 1) to build the knowledge base.


Fig1 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 1 An overview of the NICE Framework

The first step to building the knowledge base was to identify knowledge units and map them to corresponding work roles and KSAs (Knowledge, Skills, and Abilities), and then represent them in a way that is accessible by the expert system. The main issue for most cybersecurity educators attempting to use the NICE Framework to design their courses is that all the framework data are spread over a master Excel spreadsheet file, which is hard to navigate and use. Our study shows that there are three types of mappings in the framework: one-to-one, one-to-many, and many-to-many. Excel may be fine for one-to-one and one-to-many relationships; however, for many-to-many relationships, Excel produces a large number of duplicates. Accordingly, for many-to-many mappings, we used two-dimensional visual mapping techniques that map specialty areas to competencies and KSAs (see Fig. 2). This was followed by the creation of a visual mapping of the curriculum.


Fig2 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 2 Mapping of Specialty areas to KSAs

Curriculum visual mapping

The curriculum visual mapping connects knowledge units to skills and abilities based on the cross-reference of the NICE Framework and National Centers of Academic Excellence (CAE) framework. The NICE Framework lists knowledge, skills, and abilities needed to successfully complete cybersecurity tasks for students or cyber professionals. Fig. 3 shows the representation of the NICE Framework as database schema.


Fig3 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 3 NICE Database Schema

In this work, competencies in the NICE Framework are considered as assessments. Mapping specialty areas to competencies allows checking that the content discussed in a course is assessed appropriately. This many-to-many mapping is linked with KSAs, keeping the completeness in course content. Fig. 4 is the visual representation of the mapping of specialty areas with corresponding competencies.


Fig4 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 4 Mapping of a Specialty Areas to Competencies

Concept maps for knowledge representation and extraction

Concept maps provide a way to represent organized knowledge (concepts) based on a person’s understanding of a domain of knowledge.[30] They are graphs consisting of nodes (vertices) representing concepts and edges connecting these vertices. These edges may be labelled with the type of relationship between the pair of connected nodes. Concept maps have been used in many disciplines as a formal or semi-formal diagramming technique.[31] Concept mapping systems used in education and knowledge management emphasize flexibility of representation to enhance learning and facilitate knowledge capture[32], in addition to being a way to represent terminology variance, informality, and organizational variation. These factors make it difficult to match elements between maps in comparison, retrieval, and merging processes.[32] However, multiple researchers have introduced matched algorithms for the knowledge elements in educational concept maps.[32][33][34] Concept maps and expert systems are soft tools used for knowledge modelling[35], and establishing a concept map has been considered the first step in curriculum development.[36]

Concepts maps were used to model the NICE Framework, and an analysis algorithm was used to extract knowledge stored in them for use by the expert system. The relationship between concept map nodes represented the relevance or closeness of association between two components in the course being designed. This was expressed implicitly without using any labels in the visual display of the concept map; instead, the thickness of the edge was used to show the strength of a relationship. A concept map showing relationships between the different components in a course is shown in Fig. 5.


Fig5 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 5 A concept map showing strong association between two specialty areas RSK and ARC

Knowledge acquired from concepts maps and represented in the form of rules. Example facts extracted from the concept map shown in Fig. 5 are:

connect_cat_sa_sa_wt("SP", "RSK", "ARC", 0.67)

path_cat_sa_wr_ksa("SP", "ARC", "001", "K0212")

path_cat_sa_wr_ksa("SP", "RSK", "001","K059")

The above facts represent the connections between the different concepts (nodes) as follows: The first of the above facts is defined by the relation connect_cat_sa_sa_wt, where, the first parameter indicates the category name, the second and third parameters indicate the specialty areas of interest, and the fourth parameter is the strength of the connection (1 indicates maximum possible connection strength). This information is used later by the graphical user interface for visualization purpose.path_cat_sa_sa_wt is another fact that provides a full path between a category and a skill or ability. The fact takes category name as the first parameter, specialty area, work role, and KSA in the second, third, and fourth parameters respectively.

User modeling

User modeling is a subarea of human computer interaction that focuses on the process of building up and modifying a conceptual understanding of the user[37] to provide customization and adaptation to the user’s specific needs. The user model in CyberMaster aims to track the user’s interaction with the system and to build a model for user performance. For example, if the user chooses specialty area A and specialty area B to be included in their course, the user model adds this information to the user model with an indication of the strength of the connection/association between those specialty areas. The user model then provides this information to the inference engine to reason about the user’s performance and provide tailored feedback to the user.

Inference engine

The inference engine applies logical rules to the knowledge base. For example, If the connection between specialty area 1 and specialty area 2 is a weak connection, and the connection between specialty area 2 and specialty area 3 is a weak connection. Then it is deduced that the connection between specialty area 1 and specialty area 3 is a weak connection. It does also make use of the rules in the knowledge base and the user model to provide tailored feedback to the course designer.

User interface

The user interface allows the user to interact with CyberMaster. Using the interface, the user is able to explore and interact with the visual maps for the developed courses (see Fig. 6). The user interface also provides an editor, which the user can use to create or modify a course.


Fig6 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 6 Example screenshot of the graphical user interface

Course evaluation

Each new course created in CyberMaster is represented as an incremental tree (See Fig. 6) which is then transformed and saved as a concept map that holds all the course components, strength of connections, and course tags. The curriculum evaluation component in CyberMaster provides an overall evaluation for the concept map in which a final score is computed based on the strength of the connections between the different components, including specialty areas, work roles, and KSAs. This score provides a measure of how good the curriculum design is. Moreover, CyberMaster provides recommendations on how to make the design better; for example, information is provided on specialty areas and/or KSAs that can be added or removed from the current design to enhance the curriculum design (see Figure 7). The user has the freedom to apply the recommendations or leave the course as is.


Fig7 Hodhod IntJofOnlineBiomedEng2019 15-3.png

Fig. 7 An example screenshot of user feedback provided by CyberMaster

Conclusion

This paper presents work in progress to develop an expert system to guide instructors while developing cybersecurity courses in a rapid and reliable way. The system contributes to changing the current status of cybersecurity education by helping instructors anywhere in the world to develop cybersecurity courses. The culmination of this work is a robust, fully usable online curriculum development tool that can be used by both novice as well as experienced cybersecurity educators. The system evaluates every course design for quality in terms of its cohesiveness—how well the course components fit together—and conformity to a published framework for cybersecurity curriculum design. It provides a normalized course evaluation score to indicate the closeness of the curriculum to the framework. Initial test results show that CyberMaster can guide users to develop curricula following the NICE Framework for cybersecurity education. Although, the interactive and guided design process currently ensures conformity of the course content with the NICE Framework, additional cybersecurity education frameworks can be added as an option in the future.

Acknowledgements

This work is part of a project funded by an NSA CAE Cybersecurity Grant Program Grant H98230-17-0316.

References

  1. ↑ Identity Theft Resource Center (22 January 2018). "2017 Annual Data Breach Year-End Review" (PDF). https://www.idtheftcenter.org/images/breach/2017Breaches/2017AnnualDataBreachYearEndReview.pdf. 
  2. ↑ Identity Theft Resource Center (18 January 2017). "Data Breach Reports - 2016 End of Year Report" (PDF). https://www.idtheftcenter.org/images/breach/2016/DataBreachReport_2016.pdf. 
  3. ↑ Identity Theft Resource Center (2018). "Data Breaches". https://www.idtheftcenter.org/data-breaches/. 
  4. ↑ 4.0 4.1 Wang, P.; Ali, A.; Kelly, W. (2015). "Data security and threat modeling for smart city infrastructure". Proceedings from the 2015 International Conference on Cyber Security of Smart Cities, Industrial Control System and Communications: 1–6. doi:10.1109/SSIC.2015.7245322. 
  5. ↑ Wang, S.P.; Kelly, W. (2014). "inVideo — A novel big data analytics tool for video data analytics". Proceedings from the 2014 IT Professional Conference: 1–19. doi:10.1109/ITPRO.2014.7029303. 
  6. ↑ "A Human Capital Crisis in Cybersecurity: Technical Proficiency Matters". Center for Strategic & International Studies. 15 November 2010. pp. 35. ISBN 9780892066094. https://www.csis.org/analysis/human-capital-crisis-cybersecurity. 
  7. ↑ 7.0 7.1 7.2 Galston, W.A.; Baehler, K. (1995). Rural Development in the United States: Connecting Theory, Practice, and Possibilities (2nd ed.). Island Press. ISBN 9781559633260. 
  8. ↑ National Center for Education Statistics (July 2013). "The Status of Rural Education". The Condition of Education. https://nces.ed.gov/programs/coe/indicator_tla.asp. Retrieved 12 April 2017. 
  9. ↑ Schwartzbeck, T.D.; Prince, C.D.; Redfield, D. et al. (19 December 2003). "How Are Rural Districts Meeting the Teacher Quality Requirements of No Child Left Behind?" (PDF). Appalachia Educational Laboratory. https://aasa.org/uploadedFiles/Policy_and_Advocacy/files/RuralTeacherQualityStudy.pdf. 
  10. ↑ Beckner, W.; Barker, B.O. (1994). Technology in Rural Education. Phi Delta Kappa Educational Foundation. ISBN 0873673662. 
  11. ↑ 11.0 11.1 Dráždilová, P.; Obadi, G.; Slaninová, K. et al. (2010). "Computational Intelligence Methods for Data Analysis and Mining of eLearning Activities". In Xhafa, F.; Caballé, S.; Abraham, A. et al.. Computational Intelligence for Technology Enhanced Learning. Springer. pp. 195–224. doi:10.1007/978-3-642-11224-9_9. ISBN 9783642112249. 
  12. ↑ Wang, S.; Kelly, W.; Zhang, J. (2015). "Using Novel Video Indexing and Data Analytics Tool to Enhance Interactions in e-Learning". Proceedings of E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education: 1919–27. https://www.learntechlib.org/primary/p/152242/. 
  13. ↑ Wang, S.P.; Kelly, W.; Wang, X. (2015). "A Novel Threat Analysis and Risk Mitigation Approach to Prevent Cyber Intrusions". Journal of the Colloquium for Information System Security Education 3 (1): 157–74. https://cisse.info/journal/index.php/cisse/article/view/35. 
  14. ↑ Shoemaker, D.; Kohnke, A.; Sigler, K. (2016). A Guide to the National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework (2.0). Auerbach Publications. doi:10.1201/9781315368207. ISBN 9781315368207. 
  15. ↑ National Institute of Standards and Technology (August 2017). "NICE Cybersecurity Workforce Framework". NIST Special Publication 800-181. https://www.nist.gov/itl/applied-cybersecurity/nice/resources/nice-cybersecurity-workforce-framework. 
  16. ↑ Kortuem, G.; Bandara, A.K.; Smith, N. et al. (2013). "Educating the Internet-of-Things Generation". Computer 46 (2): 53–61. doi:10.1109/MC.2012.390. 
  17. ↑ Salem, A.-B.M.; Roushdy, M.; HodHod, R.A. (2005). "A Case Based Expert System for Supporting Diagnosis of Heart Diseases". AIML Journal 5 (1): 33–9. http://www.icgst.com/journals/paper.aspx?pid=P1120429002&subid=40#. 
  18. ↑ Salem, A.-B.M.; HodHod, R.A. (2002). "A Hybrid Expert System Supporting Diagnosis of Heart Diseases". Proceedings from the 2002 International Conference on Intelligent Information Processing: 301–5. doi:10.1007/978-0-387-35602-0_32. 
  19. ↑ Ragab, A.H.B.; Fakeeh, K.A.; Roushdy, M.I. (2004). "A Medical Multimedia Expert System for Heart Diseases Diagnosis & Training". Proceedings of the Second Saudi Science Conference: 31–45. https://www.kau.edu.sa/Show_Res.aspx?Site_ID=320&LNG=EN&RN=35697. 
  20. ↑ Birndorf, N.I.; Pentecost, J.O.; Coakley, J.R. et al. (1996). "An Expert System to Diagnose Anemia and Report Results Directly on Hematology Forms". Computers and Biomedical Research 29 (1): 16–26. doi:10.1006/cbmr.1996.0002. 
  21. ↑ Lee, C.-S.; Wang, M.-H. (2011). "A Fuzzy Expert System for Diabetes Decision Support Application". IEEE Transactions on Systems, Man, and Cybernetics, Part B 41 (1): 139–53. doi:10.1109/TSMCB.2010.2048899. 
  22. ↑ Filippetti, F.; Martelli, M.; Franceschini, G. et al. (1992). "Development of expert system knowledge base to on-line diagnosis of rotor electrical faults of induction motors". Conference Record of the 1992 IEEE Industry Applications Society Annual Meeting 1: 92–99. doi:10.1109/IAS.1992.244459. 
  23. ↑ Lee, H.-J.; Park, D.-Y.; Ahn, B.-S. et al. (2000). "A fuzzy expert system for the integrated fault diagnosis". IEEE Transactions on Power Delivery 15 (2): 833–38. doi:10.1109/61.853027. 
  24. ↑ 24.0 24.1 Winston, O.; Lawrence, M. (2008). "Career Guidance Using Expert System Approach". Fourth Annual International Conference on Computing and ICT Research: 123–31. http://cit.mak.ac.ug/iccir/?p=iccir08. 
  25. ↑ Daramola, O.; Emebo, O.; Afolabi, I. et al. (2014). "Implementation of an Intelligent Course Advisory Expert System". International Journal of Advanced Research in Artificial Intelligence 3 (5): 2. doi:10.14569/IJARAI.2014.030502. 
  26. ↑ Shrager, J.; Finin, T. (1982). "An Expert System that Volunteers Advice". Proceeding from the Second Annual National Conference on Artificial Intelligence (AAAI-82): 339–40. https://ebiquity.umbc.edu/paper/html/id/286/An-Expert-System-that-Volunteers-Advice. 
  27. ↑ Mackett, R.; Edwards, M. (1996). "An expert system to advise on urban public transport technologies". Computers, Environment and Urban Systems 20 (4–5): 261–73. doi:10.1016/S0198-9715(96)00020-8. 
  28. ↑ Luconi, F.L.; Malone, T.W.; Morton, M.S.S. (December 1984). "Expert Systems and Expert Support Systems: The Next Challenge for Management" (PDF). Massachusetts Institute of Technology. https://dspace.mit.edu/bitstream/handle/1721.1/47478/expertsystemsexp00luco.pdf. 
  29. ↑ Cullen, J.; Bryman, A. (1988). "The Knowledge Acquisition Bottleneck: Time for Reassessment?". Expert Systems 5 (3): 216–25. doi:10.1111/j.1468-0394.1988.tb00065.x. 
  30. ↑ Burdescu, D.D.; Mihaescu, M.C.; Logofatu, B. (2008). [http://www.iceis.org/Abstracts/2008/ICEIS_2008_Abstracts.htm "Building a decision support system for students by using concept maps"]. Proceedings from the 10th International Conference on Enterprise Information Systems: 130-135. http://www.iceis.org/Abstracts/2008/ICEIS_2008_Abstracts.htm. 
  31. ↑ Gaines, B.R.; Shaw, M.L.G. (1995). "Concept maps as hypermedia components". International Journal of Human-Computer Studies 43 (3): 323–61. doi:10.1006/ijhc.1995.1049. 
  32. ↑ 32.0 32.1 32.2 Marshall, B.; Chen, H.; Madhusundan, T. (2006). "Matching knowledge elements in concept maps using a similarity flooding algorithm". Decision Support Systems 42 (3): 1290–1306. doi:10.1016/j.dss.2005.10.009. 
  33. ↑ Feng, Y.; Goldstone, R.L.; Menkov, V. (2004). "ABSURDIST II: A Graph Matching Algorithm and Its Application to Conceptual System Translation". Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference: 640–45. 
  34. ↑ Giunchiglia, F.; Yatskevich, M.; Shvaiko, P. (2007). "Semantic Matching: Algorithms and Implementation". Journal of Data Semantics IX: 1–38. doi:10.1007/978-3-540-74987-5_1. 
  35. ↑ Baracskai, Z.; Dörfler, V.; Velencei, J. (2008). "Concept mapping and expert systems : Exploring synergies". Third International Conference on Concept Mapping: 1–5. https://strathprints.strath.ac.uk/9299/. 
  36. ↑ Chen, W.-K.; Wang, P. (2012). "A Framework of Active Learning by Concept Mapping". US-China Education Review A 2012 (11): 946–52. doi:10.17265/2161-623X/2012.11A.003. 
  37. ↑ Benyon, D.; Murray, D. (1993). "Applying user modeling to human-computer interaction design". Artificial Intelligence Review 7 (3–4): 199–225. doi:10.1007/BF00849555. 

Notes

This presentation attempts to remain faithful to the original, with only a few minor changes to presentation. Grammar and punctuation has been updated reasonably to improve readability. In some cases important information was missing from the references, and that information was added. In several cases, data was represented without a citation; an appropriate citation was added for this version. In one original reference, only the author—the NCES—is given, with no other details; an assumption was made that they were citing Chapter 3 of the Condition of Education report from 2013, which contains the statistics they reference.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:CyberMaster:_An_expert_system_to_guide_the_development_of_cybersecurity_curricula&oldid=36436"