is there any elegant way to analyze an engineer’s process?

Plenty of sentiment exists that measuring commits is inappropriate.

Has any study been done that tries to draw in more sources than commits – such as:

  • browsing patterns
  • IDE work (pre-commit)
  • idle time
  • multitasking

I can’t think of an easy way to do these measures, but I wonder if any study has been done.


On a personal note, I do believe that reflection on one’s own ‘metrics’ could be valuable regardless of (or in the absence of) using these for performance eval. I.E. an un-biased way to reflect on your habits. But this is a discussion matter beyond Q&A.

0

Not sure if you would consider it elegant, but Watts Humphrey wrote an entire book called Personal Software Process that was all about measuring your own productivity. He outlined metrics for inputs like time at your desk vs. interruptions, time spent working on various kinds of software life cycle activities, defects per amount of code. There is a technical report that gives the short version at:

http://www.sei.cmu.edu/library/abstracts/reports/00tr022.cfm

If you want to look at something like the quality of a developers code, Chidamber & Kemerer proposed several metrics for object oriented code.

Metrics for object oriented code

  • depth of inheritance tree,
  • weighted number of methods,
  • number of member functions,
  • number of children, and
  • coupling between objects.

Using a base of code, they tried to correlated these metrics to the defect density and maintenance effort using covariant analysis. Later studies did similar analysis on hundreds of Source Forge Java projects to determine their characteristics relative to CK Metrics and some additional metrics proposed later.

Metrics arising in context of Code Reviews

Defects can be categorized by many criteria:

  • severity: (major, minor, cosmetic, investigate/unknown),
  • type (logic, typo, spelling, coding standard violation, etc.),
  • origin/phase containment (requirements, design, code, etc.).

There are also preparation and inspection rates (time per reviewer, time per line of code) and defect densities (per KLOC (thousand lines of code), per minute of inspector/reviewer time).

Plotting these values against control charts can show us whether we are within bounds for the process (for example, a team that inspects 200 lines of code that finds no defects in a group that averages twenty-five defects per KLOC might be malfunctioning).

Other metrics

Other metrics that could help include those for

  • earned value to help calibrate estimation
  • velocity for those using Scrum or Kanban.
  • The work of Pareto, Juran, Demming, Ishikawa, and others from quality and industrial engineering is somewhat portable to software development with a little bit of imagination.

Limitations of Analysis

There are tremendous limits on the value of metrics. Bugs fixed per developer can mean almost anything, and when you start to punish or reward on that measurement, I bet bugs will get more numerous and granular, and the mix of difficult to easy bugs fixed will also change as the team cherry picks in the to race to have the most.

There is also a certain distraction and potentially a loss of focus and enjoyment that can come with intrusive measurement. You can’t get much more elegant (and emotionally burdened) than a lake poet like Wordsworth who said,

      Sweet is the lore which Nature brings;
      Our meddling intellect
      Mis-shapes the beauteous forms of things:--
      We murder to dissect.

While software isn’t exactly Nature, give me some latitude because I thought I would never get to use anything from high school English Literature class.

Agile might be considered a reaction to centralized, big process. Sometimes that mode of work could require so much analytical effort that the ability to reach flow while creating software all but disappeared.

4

I want to add an alternative answer that points away from standard software engineering practice toward another field with the aim of stealing from basic tools that we can adapt as needed. Quality assurance people are concerned with production, yield, and defect detection and prevention, just like software developers.

http://en.wikipedia.org/wiki/Seven_Basic_Tools_of_Quality

I like the control chart.

http://en.wikipedia.org/wiki/Control_chart

Do an activity, plot a metric, do another, plot its metric, and so on. For example, plot commits per day. The chart will scatter data that ranges from some minimum to some maximum. Perhaps later you can characterize the results to determine that when the value is low, something is impeding progress and when it is too high, work is begin done in a fast but sloppy way. How you encourage workers to speed up or slow down is left to you.

Personal metrics might be something you can correlate for yourself starting with a question like, “I feel most productive when I…”

  • Write a complete use case before start to code.
  • Write my unit tests before my code.
  • Check with stakeholders frequently to insure requirements don’t change and create massive rework on work done toward an obsolete plan.
  • Change as much code as possible.
  • Delegate well defined changes to team members who are most expert with the parts I asked them to change.
    • Give my team a complete overview, but with priorities we can finish in the current sprint.
    • Start my refactoring pass with a hierarchical list of directories, files, classes, methods, equations, variables, documentation, etc. that I will change.
    • Research a high level problem to find prior art solutions, estimating the scope and key improvements needed to make a better solution.

The old saw what we measure is what gets done might lead you to attack the problem based what you determine to be the limiting factor

or multiple factors in order of priority based on a Pareto diagram.

http://en.wikipedia.org/wiki/Pareto_chart

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật