(퍼옴) http://users.actcom.co.il/~choo/lupg/essays/becoming-a-real-programmer.html

v1.0.0

Becoming A Real Programmer (thinking about thinking...)

1. How long will it take?
2. How critical is self-criticism?
3. This is too theoretical - how about some examples?
4. Letting ideas "sink"
5. Broadening the scope of lessons learned
6. Short-cuts don't let lessons sink-in
7. Learning from other people's mistakes
8. The better programmer you become - the more humble you get
9. Being lazy the smart way
10. Conclusions

So you want to become a programmer. an independent programmer, that is not dependent on specific people to hold their hand, that can design a project and lead its development. A real programmer.

Assuming that's what you want to become, and assuming you're not there yet - how do you go about doing that?

Quite often, it will take you several years to become a real programmer. If you happen to have around you good programmers who are also good teachers, the time could shrink by much. But nothing will make this time shrink more then by you thinking about what you do, trying to develop a plan for your self-advancement, and criticizing your progress, your plan, and your changes to the plan.

If you do something, and you think you did well - you limit your ability to do it better next time - because you don't see what wasn't good enough.

If you leave self-criticism to certain occasions (such as after fully completing a given project), you miss the opportunity to learn during this project. Learning means seeing something that wasn't good enough, thinking how to do it better, trying the new way, checking if it improved something, and continuing that until you feel this isn't your worst problem any more. The longer you wait between cycles, the longer the overall process takes to converge.

Of-course, changing things all the time won't work either - cause some methods of operation only justify themselves if used for a long period of time.

Assuming that initially even usage of simple methods of operation will make us work better, we can start with using rather quick cycles, and as we make progress and master the simple methods, we can increase the time between cycles, in order to assess more complicated methods of operation.

Let us take the following example - we were told that documenting code is useful. How useful is it? How much and where to document? we need to write 'according to the book' in our first project. Leave the code for one week, and then come back and try to add a feature. Did we need to read something several times in order to remember what's going on? then lets add a comment there. Some comments were not clear? lets try to make them clear - perhaps write more? perhaps write less? perhaps improve our English grammar?

After we played with documentation for a few month we want to get to work on a larger document - something that explains the general design of a software module, for example. This takes longer to write, and it is hard to assess its quality without fully writing it, leaving that software project for a month, and then getting back to it. Our cycles slowed, because we already learned the simple things (documentation of single functions), and the more complicated methods take longer to complete, and require waiting longer in order to assess their usefulness (you don't forget the design of a module after 1 week, but you easily forget the internals of a specific function after a few days).

The method of leaving something and getting back to it later is based on two principles:

• we need to wait a while in order to be able to look at something in a new perspective - we might have got locked in one manner of thinking, and simply leaving the material for a while often helps us to unlock - because we forgot what it was that kept us locked in.
  
  
• when we need to deal with something new, we often don't fully understand it. If we leave it for a while and then come back to it, we suddenly see things we didn't see before - understand little nuances we overlooked when we were busy grasping the major principles. It could also be that we worked on a different project meanwhile, and when we got back to the original project, we saw how something we learned during the second project, could have been applied to the first project. We correlate.

When we find a mistake we made, we can simply find a fix for it and use this fix next time, or we can instead try to check if there is a more general class of mistakes, to which this specific mistake belongs. In the later case, we can find a solution to a whole class of mistakes, and thus not have to learn from different mistakes in the same class - we will avoid doing those other mistakes in the first place.

For example, suppose that we made an error of forgetting to allocate the extra null character when allocating a string in a "C" program. We coul just fix this bug and check the program again. We could think "oh, we didn't think about it", and go over all string allocations in our program to check they don't contain the same bug. We might decide that memory allocation is error-prone in general, and instead of doing the allocation and calculation of required space all over the code - we should write a function that allocates strings. We can broaden this to other types of allocations - and write functions that allocate objects of other types. If we had this problem with allocation, perhaps we have a similar problem with objects initialization - lets write functions that initialize objects, and use them, instead of writing the same initialization code all over the place. We start to see that even very small code re-use reduces the number of bugs in our code. We move from "ah, this initialization only requires 2 lines of code - why write it in a function?", to "repeating these 2 lines of code again and again will mean that if we make a mistake in 1 out of 4 locations in the code - then with 20 such code locations, we will have 5 bugs - all because we were too lazy to write a 2-line function".

See how we got from a specific memory-allocation error, to learning something about avoiding code repetition, which we would have dismissed as "a waste of time" if we were directly asked to do this in the first place.

We can argue that an experienced programmer could have shown us all of this immediately, and with enough patience, show us why something that looks trivial can save us a lot of time in the total development process. However, using such a short-cut prevents from the lesson to sink-in. We will still be tempted to choose the quicker path over using the slower path, that makes the overall journey shorter. We have to make our own mistakes in order to learn. Or we have to see how our friends made mistakes and learn from them...

When we learn of a mistake that someone else made, it is easy to think "this won't happen to me, I'm smarter" and dismiss it. But if we already invested time in seeing this mistake, why not stop, reminding ourselves we are only human and in certain situations we might do a similar (stupid) mistake? After we admit this to ourselves, we can go about learning from this mistake, as if it was our own mistake.

One possible conclusion that follows is that helping other programmers fix their own bugs can help us learn, and thus is a useful activity - not just an annoying burden.

Pride is a major obstacle for learning. "I am smart, i won't make this mistake again" is an error that will lead us to do a very similar mistake soon after. "I write good code, so i don't need to check it" will mean our simple local bugs will be hidden, and get exposed later when our module is used by other modules, which make it behave differently then our feeble tests did. By then, the bugs will be harder to track down, because the test-case is more complicated. Thus, we should reverse the cause and the result - "I check my code, and thus i write good code".

Pride will also make us think we can learn everything on our own. Given infinite time this might be true - but life is short. We want to learn faster. Let us, then, learn how to listen to other people's criticism, and weed out the useful parts in their criticism, even if we think the majority of this criticism is wrong or pointless. For example, if someone keeps telling us that we did not test our code properly, while we spent a lot of time on testing, instead of dismissing the criticism - let us listen to it. Several examples that we will be given will be wrong - we did test those parts of the code properly. But once in a while some claim for lack of testing will be correct - aha, we indeed forgot to check that input option. Now that we see it, we can start our generalization process - how do we find all possible inputs and test all of them? if there are too many possible inputs - how do we choose selected test-cases that will help identify most of the probable bugs? How do we choose test-cases in general, not just for testing inputs - what about testing timing problems?

"I will find the bugs when i debug the code" is the claim of the sloppy lazy programmer, who refuses to read their own code again after they wrote it. The smart lazy programmer will claim "if i put more time re-reading my code now when it is fresh, I will probably spend less time over-all on writing and testing this code module - if I consider the time I spend on it now, as well as during system integration and system testing later on".

The smart ultra-lazy programmer will think even further then this - "if I spend time now on learning lessons, in the long run i will be able to finish programming tasks in a shorter duration, and the time saved in the future will be much more then the extra time i spend now on my lessons learning".

In order to become real programmers, we need to think of what we plan to do. We need to think of what we just did, and of what we did a short while ago, of what we did a long while ago... Then we need to learn lessons. And then we need to broaden the lessons to more generalized classes of mistakes.

We also need to listen to other people - they might see now something that we will only see next year - even if they are not smarter then us. They are simply not locked-in by things locking us in, that we did not identify yet. Or they had the chance to learn a lesson due to a mistake they made in the past, that we didn't come across yet. Perhaps they come from a completely different background, and have completely different methodologies of thinking, that make some bug easier to spot. So we humble ourselves, and listen.

And we must avoid letting our short-term laziness hurt our much-more-useful long-term laziness.

(퍼옴) http://users.actcom.co.il/~choo/lupg/essays/software-engineering-and-uni.html

Why Do Universities Fail Teaching Software Engineering?

1. Abstract
2. What Is Software Engineering?
3. Software Engineering And Students
4. More On Students
5. More On Universities
6. How To Make It Work?
7. Financing Academic Software Projects

Many universities and other institutions try to teach software engineering concepts to their students, either as part of their regular programming courses, in courses designed to teach just software engineering, or as a set of such courses.

In this short essay, I'll supply a loose definition of "Software Engineering", show why it is so hard (or impossible) teaching it to inexperienced programmers, and try to illustrate a few methods that may be used to still put some sense of engineering into inexperienced programmers.

Software engineering is a methodological process of writing software, in a manner that can be repeated for various software projects in a high level of accuracy, and produce good software, under given time and budget constraints.

Lets explore this definition a bit. The methodology means that the software creation process is broken down into several steps, each of which is self-contained, and can be described easily, without using "magical" terms, as is common in the software industry ("i didn't know where to start, so i just hacked down something, and somehow it worked").

For example, one may break software creation into steps such as "requirements specification", "architectural design", "interface design", "detailed design", "implementation", "integration" and "testing". Each of these steps can be properly defined (e.g. "requirements specification is a step in which we define the set of functionalities our software will support, without defining yet how it will support them, or how any kind of user-interface would look like").

Good software would imply that the software does what it is supposed to do, that it has few bugs, that it can be easily maintained, and that it is relatively cheap to extend and add new features without introducing many new bugs.

The software doing what it is supposed to do might sound trivial, but in many cases the original requirements get massively cut down because the software fails to perform various operations properly. easy maintenance would mean that it would be easy for a new programmer to learn how the software works and how to fix bugs in it, and that fixing bugs does not require rewriting complete parts of the software. Easy extension would mean, for example, that the software can be easily ported to new platforms, can easily have its user interface replaced (text UI to graphical UI, or web UI) without rewriting the rest of the system, and so on.

Finally, every software development project has some limits on the amount of time until it has to be ready, and the amount of money that can be spent on the software. The engineering process should make it possible to supply a rather accurate forecast for the time and effort it will take to develop different modules of the software, using different designs. The time limits often come on expense of future maintainability and expend-ability of the software, and these issues need to be weighed against each other, and weighted appropriately, depending on the software requirements.

From the above description, one can see that software engineering introduces a lot of overhead to software development, and that this overhead might only be worthwhile for large software development. Using such process for small class exercises of few hundreds lines of code is an overkill and makes students despise or even ridicule the idea of software engineering.

Another problem with software engineering, is that it can't be practiced without understanding that small decisions have big implications on large programs. For example, suppose that a method of a communications class returns a pointer to an object it has allocated. Who should be in charge of freeing that memory? The caller of the method, or the object that initially allocated the memory? What happens if in one place its easier to have the caller free the memory, and in another place its easier to make the allocating object free the memory? an inexperienced programmer would most likely do the easier thing in each case, claiming that the simplicity of the code is important, and would overlook the importance of consistency. For a small program, it won't do much harm - since the program is written in a short time, the programmer can remember who should free the memory in each case. Even if they don't, and have a memory leak, they are not going to run this program long enough to notice - once it works, it gets dumped.

The same little decision would have a much bigger impact on a large software, that is supposed to last. A large software tends to have hundreds or thousands of classes, and not each one of them is used very often. If you had consistent rules (e.g. "the object allocating memory is always in charge of freeing it"), it will be easy to remember when to free allocated memory, and thus having less memory leaks and memory corruptions to chase.

As we saw, software engineering requires large projects to make sense. It also requires experience to make sense. In particular, bad experience - if you write software in the right way, you don't get to see how wrong badly written software can get, and thus don't learn to appreciate the right ways. Thus, part of learning software engineering is achieved by seeing how lack of it can hurt large software projects. For students "a large project" might be something they have written in one semester, in which they also studied a few other courses.
Given a software made of a few thousands lines of code, or a few tens of classes, inexperienced programmers consider the software to be a large project. Later on, when they get out to work in the industry, they will begin to realize those were in fact rather small projects. In such small projects, lack of proper engineering won't usually reveal the problems it causes, since those projects only need to work once or twice (when demonstrating them to the teacher) and don't get developed further afterwards.

Another aspect of software engineering is designing the software to be easy to maintain. Since most student projects don't get tested by end-users users, many of their bugs remain hidden, and thus the programmer isn't exposed to the amount of time it would take to debug and fix it until it becomes on the level of usable release software - something which often would take more than the amount of time it took to originally write the code and make it run. The original developer of the code tends not to stress-test their code in its weak spots, since they subconsciously know it'll cause the software to crash or malfunction - something which will require more work, that isn't really appreciated when the project is graded. Thus, the programmer cannot see how their software design affected the ability to isolate bugs in the code, and get the false impression that their design generated bug-free software.

Another aspect of learning about software engineering, is seeing how it affects the life cycle of a software project. Sometimes, experienced software engineers make decisions that look like small neat-peaking to the naked eye. Only after a while, as the project evolves to different directions, these original decisions begin to make sense to an inexperienced programmer. Some people will claim that this ability to see the need for those small decisions in advance comes with experience - and yet, this is what software engineering strives for - the ability to make these decisions using a methodological process, that generates repeated success.

Finally, there are students that understand the rules of thumb, and believe that they might be useful, but still prefer not using them for their exercises on the grounds that understanding is enough, and when they get involved in large projects, they will know how to deal with them. The problem is that by the time they get to working on large projects, they might gather bad habits and find it hard to free themselves of these bad habits. They also overlook the little things that can only be achieved with real practice, and thus will stay behind fellow students, who have spent a few years of their undergraduate studies to actually apply these rules, and gained some experience, and thus more insight.

So the conclusion we come to is that you need good familiarity and hands-on involvement in large and lasting software projects, both successful and failures, in order to grasp the essence of software engineering, and appreciate it. And in university environments (or various programming courses, for that matter) the ability to participate in such activities is rather limited.

What do universities do in order to try and teach software engineering? One thing they do is try to teach it as a set of rules of thumb, hoping that students will follow them, even if only because they are being enforced somehow when grading exercises and exams. The problem with this approach is that often the people who do the actual grading are graduate students, who themselves haven't had the chance to grab the concept of software engineering (especially if they entered graduate school directly after finishing undergraduate school).

Even if some of the teachers, or teaching assistants, do have experience with large and lasting software projects, often their students don't know about that, and hence don't trust them. When you don't trust someone, you don't listen to advise they give you if you cannot see an immediate benefit (and in software engineering, there is no benefit for very small exercises). Thus, the students actually tend to ignore their teachers' rules of thumb, seeing them as a burden, and this causes more damage than if those rules were never given in the first place.

At other times, there might be a good software engineering teacher, that indeed has experience in the field, and tries to show some real life examples to their students. These efforts, however, might be lost if the students don't get their own hands-on experience with such software. Learning software engineering on a theoretical bases, and using small code examples (since there is no time to really delve into large code examples) makes no sense, except for people who already understand software engineering in the first place - kind of a chicken-and-egg problem.

After seeing what is needed to make students appreciate software engineering, we might as well spell out a few things that will make teaching it possible. First, we need to have accredited teachers. These teachers may either be people with past or current experience in the industry, that can use it to back their claims to students. They could also be people who participated in large academic projects, that got enough credit for being large and lasting. A good example would be the work done in the MIT university, and their Athena project (see also A review of - MIT project Athena: a model for distributed campus computing). Another good example is the work done at the Washington university in Seattle, by the Distributed Object Computing (DOC) group. There exist various other such examples. The important factor is that they are large projects, involve quite a few staff members (including graduate students), and last for quite a few years, and thus carry a scope similar to that of large industrial projects.

It is also important that the students will know that their teachers have that experience. This is not to be used as a method of bragging, but rather to assure the students that their teacher is not just talking about theoretical software engineering methods; That the teacher has actually applied them, and can show them good, real-life examples, of why these methods are useful, and should be practiced even for smaller projects.

Carrying out large projects by university staff members is also good as it allows graduate students to participate in such projects, and thus be more credible to serve as teaching assistants in software engineering related courses. With good project management, it is also possible to allow undergraduate students to take part in such projects, and witness, from first hand, the complexity of such a project. When they have to delve into code created by other programmers, possibly code that is 2-3 years old, they will learn to appreciate how hard it is to get into code parts that weren't properly engineered, and how relatively easy it is to get into parts that were properly engineered. And having specific parts of the code that are badly engineered on purpose, would serve the teaching goal quite well.

Of-course, getting students involved in large software projects should be done gradually. At first, they may be introduced to small modules, learn them, and be guided in making small changes to them. It is a very useful quality to be able to delve into existing source bases, and inexperienced programmers often find it hard to do. At later phases, these students will be able to write new classes or modules. Getting credit for such projects will be more desirable than letting these students design their own software in a software project course, that will turn out to be small (they don't have enough time to work on it, and usually only 2-3 of them work on the code of such project) and having to create software that will most likely not last, and not be a part of a lasting project.

One of the major problems with carrying large software projects in universities is financing them. You need more equipment than with theoretical research, more system management staff, and more researchers than for a theoretical research.

The equipment needed is often not so hard to get as a donation from large hardware manufacturers and resellers. They already have such donation relationship with universities, sometimes donating full labs for students to work on, in a hope that these students will get used to their development environments, and endorse them when they get some influence in their future working place.

Another option is carrying software research projects that are useful for large industrial companies. Showing these companies how this research can help them, it is possible to convince them to sponsor such projects. The fact that financing a project in which graduate students and undergraduate students perform large parts of the work, would be cheaper than financing it inside the industry, it might look appealing to industrial companies. The TAO project carried by the DOC group at the Washington university and university of California, is a good example of such a relationship.

Another major problem is attracting good software engineers that both wish to carry out research work in their field, and have the skills to manage large projects at the same time. The success of such a project in fact often relies on one or more such enthusiast leaders, that carries a good reputation in the academic field, as well as in the industry. It would be easier to attract such people to the academy, if they know they will get a supportive environment, and financing to projects that will seem feasible and are personally appealing to them. Sometimes, it does not require paying them better than in the industry. The fact that they get more freedom, and without the pressure of marketing personnel, would be enough to attract a few of them to moving to the academic world.

[LUPG Home] [Tutorials] [Related Material] [Essays] [Project Ideas] [Send Comments]

This document is copyright (c) 2001-2002 by guy keren.

The material in this document is provided AS IS, without any expressed or implied warranty, or claim of fitness for a particular purpose. Neither the author nor any contributers shell be liable for any damages incured directly or indirectly by using the material contained in this document.

permission to copy this document (electronically or on paper, for personal or organization internal use) or publish it on-line is hereby granted, provided that the document is copied as-is, this copyright notice is preserved, and a link to the original document is written in the document's body, or in the page linking to the copy of this document.

Permission to make translations of this document is also granted, under these terms - assuming the translation preserves the meaning of the text, the copyright notice is preserved as-is, and a link to the original document is written in the document's body, or in the page linking to the copy of this document.

For any questions about the document and its license, please contact the author.

 How to Organize your Thesis
 Advice to Authors of Extended Abstracts
 How to write good systems paper (Loy Leven.)
 How to give a good research talk(Simon L. Jones)

논문을 어떻게 쓰는지에 대해서 알려주는 글.