amyjko / info461 Goto Github PK

Take into account the user of the application. Would be nice, links to curriculum. Could be difficult to implement this.

Andy B.

When any project uses a library, the spec should discuss what library is to be used, what version is to be used, and provide a pointer to the library and its documentation. In addition, if there are architectural concepts from that library that will be used in the project, they should state what those are (since some people just use a tiny portion of a library and others use the whole thing) and where and why they connect into the project’s architecture.

I have a thought to add extra credit to Homework 8, bug reporting. What do you think about giving +1 points for any person who files 10 real, verified bug reports. The assignment now just gives point for filing a single bug report.

Replace Medium with another competing platform (perhaps just raw HTML), and also more explicitly surface the consulting relationship and the ethics around it.

Need a policy on whether it has to be fixed. Also need feedback on grading to be able to do that well.

Need to incorporate time and expectation for students to learn APIs and frameworks. This was hidden learning in the class, and surfacing it as explicit and expected and part of class would help reduce the pressure on teammates who were the only ones with the necessary knowledge.

Some of the activities work better than others.

Given GDPR, the next offering of the class must incorporate some elements around protecting user data, offering users a way to download all their data, and a way for users to delete their data from students’ web sites.

• Who's the author? Dev? Designer?
• Who's the audience? Manager? Developer?

Need to clarify these roles to know what information should be in each document.

Sometimes the same paper shows up as an extended reading in more than one of the book chapters. In this case, students should not be allowed to select a reading to summarize that they have already summarized in a prior reading assignment.

Some just read the abstracts.

Instead, just have a section in each assignment about how to self-assess the work, and then give feedback on each submission. Then, base the entire project grade on the final deployment and defects.

@andrew Begel @zubin question regarding last question for HW5 in What practices will you follow to improve your process if it's not working? what do you mean by not working? that someone gets stuck? or when one person is not participating?

I would say that there might be a problem with process if your team is not able to achieve the goals that they had set for a particular timeline. The reasons for this can vary from people getting stuck on a difficult problem or someone not participating. Something to improve this could be to follow up with the team member or have weekly team meetings to discuss the overall progress made and difficulties faced during specific sprints. You could re-assign people, use pair programming to clarify doubts etc.

• Recording project requirements from each team on the requirements freeze date
• Weeks 1 and 2, set up time with TA to do pair grading to ensure consistency on grading

Need to provide some instruction on what makes a good report. Also elaborate on what makes a good report in the Debugging chapter of the book.

Bug bash is right before homework is due, there should be something before that to test deployment. Very few tested deployment.

This isn't really the underlying problem of course. The underlying problem is that scoping projects is hard. There could be a specific prompt to look at other's projects to understand project scope. Like a "spec bash". May or may not be graded, but at least gives visibility of range of scopes.

There have to be some good ideas from other disciplines, such as Architecture or Landscape design, where people write a lot of contracts. They might have interesting assessment ideas for checking over contracts.

Setting up development environments on each students' computer was painful, slow, and unsupported by the class.

Need some scaffolding on how to read, how to discuss. The selected readings require them to read them, but they don't know what to get from there.

Screen by screen interaction flow. Should it be required? Once they did that, it helped them realize things were missing.

In the future, instructions for the verification assignment should specify two things. 1) All tests must have a unique name or number in the document. 2) All tests should contain the unique name of the requirement specification it is testing.

Must use unique ID to make it easier to cross-referencing!

The paper midterm took too long to grade. Devise a way to run an online version in class while still preventing cheating.

Reading? Assignment?

This allows graders to keep track of requirements and traceability and feedback. We also grade on how many requirements were included, but no easy way to count.

There wasn't enough time to do it.

One person in class brought up the challenge that 2 groups are making native phone apps and one is making an Alexa app. Unless those apps are available for download, no other students in the class will be able to find bugs in them because they won't be able to run them. That's a problem. I'm debating whether to force those teams to have their apps available at the bug bash lab on May 22 and have real bugs filed against them then. @andy Ko Any ideas?

TestFlight supports Android and iOS apps without app store. Requires all email addresses of all students and a link from the application. Works for ReactNative.

Students who intended to do ReactNative, then switched to React, then had an app that required people to run things in the inspector in Chrome in mobile mode.

Some challenges with mobile development.

Reading requirements in syllabus needs to be described more clearly. Some students just didn't do it.

Consider doing it in smaller groups.

Requirements? Architecture? Testing? Need to say which element of the document is up to date. Must be able to distinguish between ambiguity or override.

Will you know if students are being truthful about summary?

You must actually read it.

Not clear if it's both an architecture specification and a functional specification or just an architecture specification. The example doesn't help, since it provides functional specification details. Moreover, there's no functional specification homework assignment, so it's not clear when students would think about the functional spec.

• Ask an expert
• Get feedback on an idea
• Reuse ideas from others
• Delegate

Also, frame the entire class as one big team.

Some teams didn't utilize the architecture documents they created earlier in course. In some cases, this was reasonable, since they were first drafts and really first attempts at thinking about architecture. In other cases, it was problematic, since teammates relied on those documents to understand the code, but the architecture in the documents was abandoned.

Many teams found GitHub's wiki unfamiliar to too difficult to edit, so teams relied on Google Docs instead. This created extra work, because we needed the wiki and it's history to grade, forcing the students to duplicate effort.

It's waterfall, because there's only one sprint. Could do two two week sprints instead and there's a deliverable by the end of week 8.

Need to explain:

• Comprehending code is hard and takes a lot of time
• It requires gathering evidence in code and documentation to form theories and make inferences from evidence
• In large companies, new employees often spend hundreds of hours comprehending code before they are allowed to make changes.
• Goal of the lab is to help normalize this uncertainty and start developing some new skills to comprehend code.

There's a better structure I invented today:

• Explain how honesty connects to coordination
• Model being vulnerable by sharing a personal story of mine
• Have people mimic that behavior with their team, sharing their own stories
• Teaching a practice of doing this in standup meetings and how managers should model it
• Then talk about how to respond to unsafe environments with courage
• Have them reengage with their team on the architecture assignment

Allows for iteration, more learning. Can get up to 75% of points they lost. They also can lose more points on the new content.

Andy B liked prose.

Clarify in Canvas which reading it is.

I ran some stats on the midterms. Average score was 17/20. Questions were answered correctly by an average of 85% of students. Question 9 was only answered correctly by 29% of students, followed by Question 11 (55%), Question 13 (61%), Question 4 (71%), and Question 18 (77%).
Oops, Question 9 above was actually Question 7.
Question 7 was new this year. “A set of requirements is deemed precise if it can be validated by formal methods.” Answer FALSE.
Question 11 was “Static typing is a language feature that checks for type errors at runtime.”
Question 13 was “Prototypes are central to Boehm’s Spiral Model.”
Question 18 was new this year. “Bug reports from customers are fixable only when accompanied by a stack trace.” Answer FALSE.
Q7: People believed there to be a false equivalence between precision and formal methods.
Q11: Probably people didn’t equate the word runtime with running the program.
Q13: Just a memorization problem.
Q18: It seems like it should be true, but developers are pretty smart and can often find bugs without stack traces.
Generally the problems stem from poor understanding of logical chains of reasoning, especially when we wrote the converse or inverse of something true on the test question.

Some students used CSE 154 as their web development pre-requisite, which didn't prepare them with any version control or web development framework knowledge.

A flaw in the verification spec assignment is that if they haven't yet started coding, they won't be able to tell us what their unit tests are going to be. Are we forcing students to use TDD?

Students don't know anything about automated testing, and we don't teach it.

Some people are doing more difficult things, and so they have more opportunity to lose more points.

• Start with discussion of framework use and rationale for choice
• Then discuss how this works in industry
• Then introduce how adaptation is a key quality, why it's hard to adapt
• Then introduce activity, framing it as argument for and against current choice.
• Wrap with discussion of how these decisions play in industry

Could have them write alumni.

If the same kind of defect appears in a specification more than once, you only get dinged for it once.

Homework 6 - verification should have an addition to the grading rubric: -0.1 for writing down a test that is testing functionality not mentioned in the spec.