How Could AI Support Design Education? A Study Across Fields Fuels Situating Analytics
2404.17390
0
0
🤖
Abstract
We use the process and findings from a case study of design educators' practices of assessment and feedback to fuel theorizing about how to make AI useful in service of human experience. We build on Suchman's theory of situated actions. We perform a qualitative study of 11 educators in 5 fields, who teach design processes situated in project-based learning contexts. Through qualitative data gathering and analysis, we derive codes: design process; assessment and feedback challenges; and computational support. We twice invoke creative cognition's family resemblance principle. First, to explain how design instructors already use assessment rubrics and second, to explain the analogous role for design creativity analytics: no particular trait is necessary or sufficient; each only tends to indicate good design work. Human teachers remain essential. We develop a set of situated design creativity analytics--Fluency, Flexibility, Visual Consistency, Multiscale Organization, and Legible Contrast--to support instructors' efforts, by providing on-demand, learning objectives-based assessment and feedback to students. We theorize a methodology, which we call situating analytics, firstly because making AI support living human activity depends on aligning what analytics measure with situated practices. Further, we realize that analytics can become most significant to users by situating them through interfaces that integrate them into the material contexts of their use. Here, this means situating design creativity analytics into actual design environments. Through the case study, we identify situating analytics as a methodology for explaining analytics to users, because the iterative process of alignment with practice has the potential to enable data scientists to derive analytics that make sense as part of and support situated human experiences.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- The paper explores how design educators' assessment and feedback practices can inform the development of AI systems that support human experiences.
- It builds on Suchman's theory of situated actions, and presents a qualitative study of 11 design educators across 5 fields.
- The study identifies key themes: design process, assessment and feedback challenges, and computational support.
- The paper introduces the concept of "situating analytics," which aligns AI-powered assessment and feedback with actual teaching practices.
Plain English Explanation
The researchers in this paper wanted to understand how design educators provide assessment and feedback to students, and use those insights to develop AI systems that can support human experiences. They conducted a study of 11 design educators across 5 different fields, and identified key themes around the design process, the challenges of assessment and feedback, and how technology could potentially help.
The researchers build on the idea of "situated actions," which means that human activities are deeply tied to the specific context and environment in which they take place. Similarly, the researchers argue that for AI systems to be truly useful, they need to be aligned with the actual practices and needs of the people using them. This led them to propose the concept of "situating analytics," which means integrating AI-powered assessment and feedback tools directly into the design environments where they will be used.
For example, the researchers developed a set of "design creativity analytics" - metrics like Fluency, Flexibility, and Legible Contrast - that can provide students with real-time feedback on their design work. But the key is that these analytics are not just generic measures, but are specifically tailored to fit the situated needs and practices of design education. The iterative process of aligning the analytics with actual teaching and learning is what the researchers call "situating analytics."
The goal is to create AI systems that seamlessly integrate with and support human experiences, rather than imposing a technological solution that doesn't fit the context. By deeply understanding the situated practices of design educators, the researchers aim to develop AI tools that are truly useful and meaningful.
Technical Explanation
The paper presents a qualitative study of 11 design educators across 5 different fields, including architecture, industrial design, and interaction design. The researchers used interviews and observations to identify key themes around the design process, the challenges of assessment and feedback, and how computational support could potentially help.
Building on Suchman's theory of situated actions, the researchers argue that for AI systems to be useful, they need to be aligned with the actual practices and needs of the people using them. This led them to propose the concept of "situating analytics," which means integrating AI-powered assessment and feedback tools directly into the design environments where they will be used.
As part of this, the researchers developed a set of "design creativity analytics" - metrics like Fluency, Flexibility, Visual Consistency, Multiscale Organization, and Legible Contrast - that can provide students with real-time feedback on their design work. Importantly, these analytics are not just generic measures, but are specifically tailored to fit the situated needs and practices of design education.
The researchers also invoke the "creative cognition's family resemblance principle" twice in the paper. First, to explain how design instructors already use assessment rubrics, and second, to explain the analogous role for design creativity analytics: no particular trait is necessary or sufficient, but each one tends to indicate good design work. The key is that human teachers remain essential, and the AI-powered analytics are designed to support, rather than replace, their efforts.
Critical Analysis
The paper presents a thoughtful and well-designed study that offers valuable insights into how design educators approach assessment and feedback, and how AI systems could potentially support those practices. The researchers' focus on "situating analytics" is a particularly compelling approach, as it underscores the importance of aligning technological solutions with the specific needs and contexts of the people who will use them.
That said, the paper does not delve deeply into potential limitations or challenges. For example, it does not address issues around data privacy, algorithmic bias, or the potential for over-reliance on AI-powered feedback. Additionally, while the researchers develop a set of "design creativity analytics," they do not provide a detailed evaluation of their effectiveness or how they compare to other existing tools.
Further research could explore these areas in more depth, as well as investigate the broader implications of "situating analytics" for other domains beyond design education. It would also be interesting to see how the researchers' approach might evolve as AI systems become more advanced and capable of more nuanced, contextual understanding.
Overall, the paper presents a compelling and thought-provoking contribution to the field of human-AI interaction, and its focus on aligning technology with situated human practices is a valuable perspective that deserves further exploration.
Conclusion
This paper explores how design educators' assessment and feedback practices can inform the development of AI systems that support human experiences. By conducting a qualitative study of 11 design educators, the researchers identified key themes around the design process, assessment and feedback challenges, and the potential role of computational support.
The researchers' concept of "situating analytics" is particularly compelling, as it underscores the importance of aligning AI-powered tools with the specific needs and practices of the people who will use them. By developing design creativity analytics that are tailored to the situated context of design education, the researchers aim to create AI systems that seamlessly integrate with and support human experiences.
While the paper does not address all potential limitations or challenges, it offers a valuable perspective on how to make AI useful in service of human experiences. By deeply understanding the situated practices of design educators, the researchers demonstrate how AI can be developed to truly fit the needs and contexts of its users, rather than imposing a technological solution that doesn't quite match the reality of human activity.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Indexing Analytics to Instances: How Integrating a Dashboard can Support Design Education
Ajit Jain, Andruid Kerne, Nic Lupfer, Gabriel Britain, Aaron Perrine, Yoonsuck Choe, John Keyser, Ruihong Huang, Jinsil Seo, Annie Sungkajun, Robert Lightfoot, Timothy McGuire
0
0
We investigate how to use AI-based analytics to support design education. The analytics at hand measure multiscale design, that is, students' use of space and scale to visually and conceptually organize their design work. With the goal of making the analytics intelligible to instructors, we developed a research artifact integrating a design analytics dashboard with design instances, and the design environment that students use to create them. We theorize about how Suchman's notion of mutual intelligibility requires contextualized investigation of AI in order to develop findings about how analytics work for people. We studied the research artifact in 5 situated course contexts, in 3 departments. A total of 236 students used the multiscale design environment. The 9 instructors who taught those students experienced the analytics via the new research artifact. We derive findings from a qualitative analysis of interviews with instructors regarding their experiences. Instructors reflected on how the analytics and their presentation in the dashboard have the potential to affect design education. We develop research implications addressing: (1) how indexing design analytics in the dashboard to actual design work instances helps design instructors reflect on what they mean and, more broadly, is a technique for how AI-based design analytics can support instructors' assessment and feedback experiences in situated course contexts; and (2) how multiscale design analytics, in particular, have the potential to support design education. By indexing, we mean linking which provides context, here connecting the numbers of the analytics with visually annotated design work instances.
4/9/2024
🤖
Crafting Tomorrow's Evaluations: Assessment Design Strategies in the Era of Generative AI
Rajan Kadel, Bhupesh Kumar Mishra, Samar Shailendra, Samia Abid, Maneeha Rani, Shiva Prasad Mahato
0
0
GenAI has gained the attention of a myriad of users in almost every profession. Its advancement has had an intense impact on education, significantly disrupting the assessment design and evaluation methodologies. Despite the potential benefits and possibilities of GenAI in the education sector, there are several concerns primarily centred around academic integrity, authenticity, equity of access, assessment evaluation methodology, and feedback. Consequently, academia is encountering challenges in assessment design that are essential to retaining academic integrity in the age of GenAI. In this article, we discuss the challenges, and opportunities that need to be addressed for the assessment design and evaluation. The article also highlights the importance of clear policy about the usage of GenAI in completing assessment tasks, and also in design approaches to ensure academic integrity and subject learning. Additionally, this article also provides assessment categorisation based on the use of GenAI to cultivate knowledge among students and academic professionals. It also provides information on the skills necessary to formulate and articulate problems and evaluate the task, enabling students and academics to effectively utilise GenAI tools.
5/6/2024
🏷️
The Interplay of Learning, Analytics, and Artificial Intelligence in Education
Mutlu Cukurova
0
0
This paper presents a multi dimensional view of AI's role in learning and education, emphasizing the intricate interplay between AI, analytics, and the learning processes. Here, I challenge the prevalent narrow conceptualization of AI as stochastic tools, as exemplified in generative AI, and argue for the importance of alternative conceptualisations of AI. I highlight the differences between human intelligence and artificial information processing, the cognitive diversity inherent in AI algorithms, and posit that AI can also serve as an instrument for understanding human learning. Early learning sciences and AI in Education research, which saw AI as an analogy for human intelligence, have diverged from this perspective, prompting a need to rekindle this connection. The paper presents three unique conceptualizations of AI in education: the externalization of human cognition, the internalization of AI models to influence human thought processes, and the extension of human cognition via tightly integrated human-AI systems. Examples from current research and practice are examined as instances of the three conceptualisations, highlighting the potential value and limitations of each conceptualisation for education, as well as the perils of overemphasis on externalising human cognition as exemplified in today's hype surrounding generative AI tools. The paper concludes with an advocacy for a broader educational approach that includes educating people about AI and innovating educational systems to remain relevant in an AI enabled world.
4/8/2024
👀
Student Reflections on Self-Initiated GenAI Use in HCI Education
Hauke Sandhaus, Maria Teresa Parreira, Wendy Ju
0
0
This study explores students' self-initiated use of Generative Artificial Intelligence (GenAI) tools in an interactive systems design class. Through 12 group interviews, students revealed the dual nature of GenAI in (1) stimulating creativity and (2) speeding up design iterations, alongside concerns over its potential to cause shallow learning and reliance. GenAI's benefits were pronounced in the execution phase of design, aiding rapid prototyping and ideation, while its use in initial insight generation posed risks to depth and reflective practice. This reflection highlights the complex role of GenAI in Human-Computer Interaction education, emphasizing the need for balanced integration to leverage its advantages without compromising fundamental learning outcomes.
5/3/2024