ITCS 6010/8010: AI-Driven Trustworthy Software Development
AI-Driven Trustworthy Software Development
Welcome to ITCS 6010/8010. This project-based graduate course explores how software engineering practices must evolve when Artificial Intelligence (particularly Large Language Models) becomes an active participant in the software development process.
In this course, we will build a real software system using AI tools across the software lifecycle, including requirements engineering, design, coding, testing, deployment, and monitoring. The focus is not only on learning how to use AI tools in software development, but also on understanding how trust in software must be engineered, evaluated, and defended when development artifacts are generated by probabilistic and fallible AI systems.
Course Description
This course provides a project-centered exploration of software development in the presence of Artificial Intelligence (AI), with a particular focus on trustworthiness and evidence-based engineering. As AI systems, especially Large Language Models (LLMs), are increasingly being used for requirements engineering, design, coding, testing, and deployment, traditional assumptions about correctness, reliability, and developer intent no longer hold. This course examines how software engineering practices must adapt when development artifacts are probabilistic, fallible, and partially autonomous.
The course guides students through the full software development lifecycle using a single evolving project. Students work in teams to design, implement, and evaluate a realistic software system in which AI tools actively generate or influence development artifacts. Rather than treating AI as a productivity aid, the course treats AI as a first-class development actor whose outputs must be questioned, constrained, and evaluated. Central topics include AI-assisted requirements elicitation, AI-influenced architectural design, AI-generated code and tests, and the risks introduced by AI-generated configurations and deployment artifacts.
A central principle is the use of a DevOps pipeline as an enforcement and evaluation mechanism. The relevant trustworthiness properties, such as reliability, security, safety, and robustness, are evaluated continuously through explicit trust gates integrated into the pipeline. Students explore how to design systems so that trust claims are observable, testable, and defensible, and to document failures through incident analysis rather than hiding or avoiding them.
Learning Outcomes
Upon successful completion of this course, students will be able to:
- Understand and explain how the use of Artificial Intelligence, particularly Large Language Models, fundamentally changes software development practices and challenges traditional assumptions about correctness, reliability, and developer intent.
- Apply AI-driven techniques across multiple phases of the software development lifecycle, including requirements engineering, design, coding, testing, deployment, and monitoring, while recognizing the risks and limitations introduced by probabilistic AI-generated artifacts.
- Design and implement a DevOps pipeline that integrates AI-assisted development tools and enforces trustworthiness properties through explicit, observable trust gates.
Course Topics
- Getting Started: Artificial Intelligence, particularly Large Language Models, as active participants in software development, and the implications of probabilistic, fallible AI-generated artifacts for responsibility, correctness, and engineering practice.
- Trustworthiness as an Engineered Property: Foundations of software trustworthiness, including reliability, safety, security, robustness, and the distinction between trust, confidence, and correctness.
- AI-Assisted Requirements and Design: Use of AI for requirements elicitation and architectural design, identification of ambiguity and inferred behavior, and design principles that enable evaluation and enforcement of trust.
- DevOps Pipelines for Continuous Trust Evaluation: Continuous integration and delivery pipelines as mechanisms for enforcing trust, generating evidence, and making AI-related risks observable throughout the software lifecycle.
- Code, Testing, Deployment, and Runtime: Delegation of coding and testing tasks to AI systems, limitations of testing and oracles, security and deployment risks, and runtime monitoring for detecting silent or evolving failures.
- Adversarial Analysis and Evidence-Based Trust Claims: Adversarial testing, trust-gate refinement, peer auditing, incident analysis, and communication of defensible trust claims using a claim-evidence-limitation structure.