last updated:06 Aug 2003 15:16 UK time JOS Statistics - Recent Comments (Comments added for week ending Sun 05 Jan 2003) | View Other Weeks Number of Jobs by Programming Language | Sun 05 Jan | Matthew Lock Anyone noticed this over at Slashdot? The methodology is a bit dodgy but food for thought if you are looking for a job: Number of Jobs by Programming Language http://www.bitbreather.com/programming_languages.html Sun 05 Jan | crusty admin | I kept track of this awhile back and in early 2001 there were 24,000 listings for java. Sun 05 Jan | Dan Maas | I'd never hire someone or expect to be hired just for knowing language X... If a person can't pick up any unfamiliar language within a week, they're not worth hiring. Unless you are just looking for a warm body... Sun 05 Jan | Troy King | Dan, that was spoken like a true comp-sci sophomore. If you honestly beleive any programmer can master an unfamiliar language in a week, enough to be productive and write quality code, you're obviously a beginner. Learning basic syntax in a week is possible, but mastering even the simpler languages, and learning their ins and outs, not to mention the libraries, takes a lot longer than a week, and anyone who's actually worked as a programmer realizes that. Sun 05 Jan | clumps | I dunno troy, I have never really even 'known' any of the languages I've worked with before being put on the job. I learned C/C++ on a sysadmin job I had in college, and every other job , where I've worked with java, perl, or python, I've just said 'i know C++' and got hired. It maybe took more than a week to learn the basic language, but not much. People have these weird arguments about 'mastery', but really, you should only learn enough to get the job done. There is no reward to becoming a guru, in most cases... Sun 05 Jan | Stephen Jones | Clumps, Are you working as a sysadmin who needs to use certain scripting or programming languages occasionally or as a progammer working full time producing code in them; there is a difference Sun 05 Jan | clumps | i've been a programmer since the college sysadmin job where i learned C++. Sun 05 Jan | Wayne Venables | Learning a language is easy... learning the ins and outs of whatever libraries are tied to the language is another story altogether: For C/C++ you have the STL (which very few people are all that good at) and the API for whatever OS you are using (also tricky stuff). For Java, you have the Java class library as well as the countless number of additional frameworks. This also goes for C# and the .net experience. For Visualbasic, you have the VB API (classes, controls, etc) and the various DB libraries (ADO, DAO, etc) which have their strange quirks and issues. ...and on and on... Sun 05 Jan | Li-fan Chen | Every job offer in the world ask for Java and VB, and most recruiters add C++ because well 'it seems to net the really smart ones.' And oh yeah, 'you must have 20 years Java experience'. I hope everyone understands that one ought to read this report with a (meteor-size) grain of salt. Sun 05 Jan | T. Norman | To be a guru, you generally do need a few years working with the language AND the particular libraries and frameworks associated with the language. So a Java guru would not only know Java's syntax, but would also know Swing, AWT, EJB, RMI, JSP, JDBC, and JNDI. You don't learn all of that well in a few weeks. However, while they write the ads as if they want a guru, when they actually hire somebody (who probably had to lie and claim that he/she had 5 years of C#) based on the ad, that person is paid only $50-$60K and writes programs based on specifications handed down by a senior employee. Gurus are almost never hired from an Internet job ad. Either they are already there in the company, or they get a full-time offer after being hired as a contractor. Or thru networking. On the other hand, there is some small justification in looking for specific languages even for non-guru programmers because there are so many programmers who are miserable at learning a new language, because they failed to acquire the conceptual background (whether on the job, self-taught or in school) to do anything beyond the specific language they have been using. Somebody from a purely COBOL background, or whose only experience is a Visual Basic job which they got hired for a few years ago without any computer science training (school or otherwise), may not have the background to understand pointers and object-oriented principles, and is likely to have a very hard time working on a C++ or Java project. Observations at my company have confirmed this problem. But still, it is the conceptual background and general 'smartness' and enthusiasm that will matter more than the specific language. Companies should look at the conceptual background that the candidate's experience covers, rather than a letter-for-letter match of programming languages. A Smalltalk expert who has been writing programs for DNA analysis is probably not going to have any trouble picking up Java. I remember a couple years ago it was a long-time Smalltalk programmer who won a JavaWorld programming contest with his first Java program. Unfortunately, the HR drones who post these ads and scan resumes also don't have the conceptual background to understand that Delphi experience would provide a good foundation for Visual Basic, or that C++ experience with CORBA would translate well to Java and EJB. Sun 05 Jan | Samuel | I've worked with several guys that were able to learn a new language in a week. At least they thought they had learned it. :/ Sun 05 Jan | John Topley | Wither Delphi? And I've never heard of Scheme... Sun 05 Jan | rally monkey | I don't understand how somebody can claim to be a programmer yet never have heard of Scheme. You ought to be flogged. Sun 05 Jan | A Software Build Guy | Scheme is a LISP varient developed for advance AI research during the late 1960's and 70's (at MIT I believe). I believe Rice University developed this free Scheme learn enivronment: http://www.drscheme.org/ Cheers MAD Sun 05 Jan | Dan Maas | When I say 'pick up a new language within a week' I don't mean achieve complete guru-level mastery; but I believe one should at least be able to track down problems with a debugger, etc... Also I assume that applicants should be *expected* to already have exposure to each major field of programming languages (e.g. one from the C/C++/Java/C# family, one from the Python/Perl/Tcl family, one from the LISP family, one CPU assembly language, one database query language, etc)... With that background it should not be difficult to adapt to an unfamiliar member of the same families. (a few weeks ago I was asked to fix a broken Java program; even though I'd never used Java before, my experience with C and C++ helped enough that I got it done in a few hours). Sun 05 Jan | ODN | It's okay if you have never heard of Scheme. It's okay if you are only conversant in one 'world' of programming language/environment. Each world is self-sufficient enough these days that there is no fundamental need to cross over into the other worlds. In fact each world is so large that once you've made a sizable commitment to one world, it requires some serious motivation (your career path requires it, or shooting for guru status) that most people don't have the energy nor time to dig deeply into the other worlds. Nobody can truly be a guru these days in all the software worlds out there. You don't need to know one Germanic, one Romance, one Slavic, and one Finno-Ugric language to write classic literature. Sure, knowing Latin might come in handy (depending on your chosen language), but it's not a requirement. Now if you're a linguist, that's another matter, but how many of those are there compared to authors? Besides, most linguists are doing research, not writing novels. So if someone doesn't know the same exact industry/academia factoids as you do, don't bash them. There are many paths to the same level of general computing knowledge, and your particular path is more likely than ever these days to deviate significantly from the next person's path. Even if it feels like you've focused your efforts only on the need-to-know subject areas, I'll bet that someone out there has a different definition of need-to-know that is just as valid. And if neither one of you can lay your egos and single-world-view aside, you will both look down on each other, and no amount of debate will persuade you to change your mind. Sun 05 Jan | Bill Carlson | Golly, Dan, I guess I'm only 3% as smart as you. It's been 9 months with C#/.NET and I'm only now comfortable with my understanding of best practices, performance considerations, interop, the 6,500 CLR classes, deployment and debugging. Think of how much better off we'd be if we hired the 'one week guru'. Maybe I'll cure cancer next week. Should only take me 3 or 4 days to learn the basics of cellular biology. Nah, Middle East peace sounds interesting. No more than a week to study the region's layout. No, perhaps supercomputing. After all, there's only N and P silicon. What idiot can't figure that out? How about I just E-Mail you the 600 page spec to our app and you can send us a binary? No later than Thursday. I told marketing they can believe in you. Sun 05 Jan | Bill Carlson | Appologies Dan, for the last post. I didn't see your clarification. Sorry, all! Sun 05 Jan | HeyMacarana | I'd take those numbers with a grain of salt since the actual number of jobs will probably significantly vary. Not a large majority of companies use only those two job boards (Monster and Dice), and many don't even fill positions through online means at all. Instead most of the openings are filled through current employees who either switch over to the new positions or recommend a friend or family member to the HR dept. for hiring. So those positions rarely even make it to the message boards. C++ Project Documentation Tools | Sat 04 Jan | Joe Can anybody suggest some good automatic documentation tools for VC++/C++ projects? I have been doing the documentaion manually so far and I like it that way. But I want to see how useful these tools are. Thanks in advance. Sun 05 Jan | Matthew Lock | Check out Doxygen http://www.stack.nl/~dimitri/doxygen/ Sun 05 Jan | Mike Swieton | I'll second doxygen. Powerful enough to get the job done, but doesn't necessitate any sort of syntax (As I recall, which may be wrong, Javadoc needs at least a little special comment syntax). I think doxygen should be enough for most projects. does programming make you dumber? | Sat 04 Jan | worried Back in college, I was pretty smart. I could sleep through all my math classes,and get As (i have a math degree.) After college I needed some money, so I became a programmer (maybe I wasnt so smart...) I have been programming for about five years, and got bored, so I took an introductory cellular and molecular biology course at the local university. I barely made it through the course. I would think I would know the material, but would get totally destroyed on the exams. I didnt get destroyed on the exams because I didnt know the material - I studied about 3 or 4 hours a day. I got destroyed because I couldnt apply my knowledge of the material to the tricky questions. I.e. use critical thinking skills. This was particularly worrisome because I thought that critical thinking skills were what I was developing all day at my programming job. Ive never felt so stupid in my life. I have two theories as to why I sucked: 1. Im old (28) and just am not as smart as I used to be 2. the task switching problem - working all day on something not really related to bio, made it hard to switch to bio at night Has anyone else experienced this? Sat 04 Jan | Dean Wormer | First 'programming' means different things to different people. Second, at 28 you're probably just not onto studying as much as in college. You probably think you slept through your classes but I bet you actually put forth alot of effort...(maybe it's nostalgia) Sat 04 Jan | X. J. Scott | I don't have a good feelingh about either of your two hypothesis. Is it possible that because you are now successful that you thought it wouldn't be too hard and just didn't apply yourself enough? Is this a problem that has repeated in several classes or just one you have taken. Don't forget hat in college, you were at the tail end of 12 years of having learned how to study and got to classes. Now you have spent more time working and designing and being creative which is different from schooling. I suspect that if you took a few classes you'd get back in hte saddle. Of course your familiarity with the prerequsites of the class you took are further in the past now than they were when you went through the first time. Sat 04 Jan | Julian | Coursework (attending lectures, reading textbooks, doing homework, taking tests) requires different kinds of thinking than programming for a living. Your mindset and way of tackling problems has changed to an approach that's more helpful in your workplace but less conducive to taking classes. That doesn't make you any dumber. Sun 05 Jan | Mark Roden | I've done both intro to molecular bio courses and programming courses. Almost everyone in the bio courses who were programmers or engineers failed miserably, because they kept trying to apply mathematical-style rules to biology. Those rules only start to appear at a much higher level than intro courses. Remember that a biologist is a chemist who failed math; it's not that you're dumber, it's that you're thinking too hard and applying way too much analytical thinking to what is essentially a regurtiation course. Sun 05 Jan | anon | What sort of programming do you do, and what do you think of it? Not all programming is the same. Sun 05 Jan | worried | I do a combination of writing data clustering algorithms for microarray data sets (in perl) and typical 'enterprise' style data management apps (display data in different ways on web site). In a previous job I did mostly 3-D simulations of human motion (dump a bunch of datapoints into an openGL model, watch the animation) . I took the bio course because I now work in a genetics lab and thought it would be useful. I also thought I could maybe use it as a prerequisite for medical school, but I no longer think that... :'-( The main reason I'm worried is that I think I have the regurgitation down; my issue was when the test had questions where I actually had to think, I got screwed. However, someone else told me that my main problem was that I was 'visualizing' the concepts. Like for instance, I would remember 'mitosis' by seeing each step in the process. however, this guy told me that was way too slow, and what I needed to do was just do rote word association for each step. Sun 05 Jan | Stephen Jones | 'Remember that a biologist is a chemist who failed math' Actually I've just bought two math books, both because they are relevant to my side interests, programming and evolutionary biology. The first is a pre-requisite for studying Computer Science; the second is a prerequisite for studying biology (in particular ecology). I can assure you that the math for the programming course is more basic than the math for the biology course (in case anybody wonders why ecologists need maths the answer is population demographics). With regard to 'worried's' question I suspect that it is the adaption to a different type of thinking that is causing the problem. Ask yourself why so many people who are really bright at other things are pretty hopeless at Math, and you will realize that Math thinking is possible a specialized skill. If you're thinking everything through and that's taking too much time, don't bother. Just repeat the classes; you'll make the time up later. And, no, programming won't make you dumb (which will rather kybosh your chances of becoming a manager!). If it did, then after twenty-five years of teaching - which is even more mind-deadeining - I'd be in danger of being mistaken for a concrete post. Sun 05 Jan | X. J. Scott | 'Ask yourself why so many people who are really bright at other things are pretty hopeless at [Subject Y]' This is really interesting since it could explain some of the situations where someone who appears to be quite brilliant becomes dumbfounded when faced with programming a VCR or setting up their browser settings or even simpler computer tasks. The computer model just doesn't fit their pattern of thinking. Sun 05 Jan | Hello | My dad is a doctor and a medical school lecturer, and he was telling me that when he uses the computer, he actually has to think out steps like this: drag mouse device across mousepad mouse moves pointer thing on screen make pointer thing move to bottom left hand corner of screen click one of the buttons when pointer thing is over box etc... i was sort of amazed by this. my mom, who is a nurse and thus has to spend a lot of time in front of the box doing clerical tasks, has already internalized all that. I'm assuming it has a lot to do with daily repetition. I couldn't drive stick at all until I bought a stick automobile and had to do it everyday... Sun 05 Jan | Adam Young | I found that I had problems on the loigic portion of the GRE. One person hypothesized that it was becasue I treated the problems like a debuggin problem. I program slowly and cautiously, because I have burned many times by one little thing out of place; copy paste errors, etc. My approach to problem solving has slowed down. See, in a test you care about how many you get right in a certain amount of time. IN programming, you care about solving the problem, and you don't move on until you do. Time crunches are made easier by going faster, you just make more mistakes and slow down getting the job done. It is a difference in attitude and focus. You are probably a better problem solver than you were in School, but on in the realm of coding. Sun 05 Jan | a. hole | Adam, did you have a problem with the logical reasoning or the logic games? I took the lsat and the logic games kicked my arse. Once I learned the methods it became a lot easier. I used the logic games bible: http://www.powerscore.com/ Now I understand that they've done away with the logic games on the new GRE. CityDesk Entity Classes - locking strategy? | Sat 04 Jan | Albert D. Kallal Ok, just taking a quick read of Joel’s class objects on CityDesk, I see nothing about a record locking strategy. What happens when two users try to edit/load the same article? I of course also tend to create a set of objects for most applications that handles complex udaates..and of course more importantly centralizes the update code. I also usually add: MyObject.MyLock The above is a Boolean, and thus: MyObject.LoadId = 222 So, now we have a record id of 222 set. The property LoadId of course can thus be both set, or viewed. When doing some critical operations, I do If MyObject.MyLock = true then ‘ bla bal al else ‘ ok..user cancled end if Note that IF the object/record entity in this case IS LOCKED, then I pop up a nice screen showing the user name, and workstation (pc name) that currently has the lock. It also has a timer event, and if the user that is holding the lock exits, then the above small prompt screen automatically closes and continues (the above will thus return true). However, there is a cancel on that small “locked” prompt form. If the user hits cancel, then the above MyLock thus returns false. Joel: In your examples you did not mention any kind of record/article locking strategy? Do you have a record locking strategy along with your objects/Entity examples? Any neat ideas…or have you dealt with this problem yet? To everyone: What do most of you folks do out there for record lock type strategy? Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sat 04 Jan | Adam Young | OK, brief summary of Transaction Requirements: ACID: Atomic, Consistant Isolated,Durable Locking Does not get you Atomic, as your code needs to do that. Locking does not get you consistant, that has to be enforced elsewhere Locking gets you Isolated. IE, I can't get at your record until you are done with it. Locking does not get you Durable, as your changes may be lost dues to system failure etc. If you want all of this, use a transaction monitor. JBoss for EJBs is free and does this, but you have to use entity beans to get it. Microsoft Transaction server does this for MS technologies. At least, that was the technology last I looked. Only problem there is that it had to be stateless. Locking without serious thought can lead quite easily to deadlocks. I know, I do it all the time. Sat 04 Jan | Albert D. Kallal | Well, we can assume that the data engine we are using has transactions. It solves nothing from a user interface point of view does it? Two people want to update the same word document. You have to serialize this process. We can get into a big discussion about how atomic a value is here, but that is not really our problem. In general, we do NOT have the technology to update partial parts of a word document. The same goes for the average application. It is not a record, but usually a group of records that we must deal with. So, we are stuck back to square one…and ACID means nothing again!! City desk is a document management system, and that is what the examples we are talking about apply to. However, my example was in fact a tour object from a reservation system (been writing those things for 10 years now). Even in my case (a tour reservation) object, this issue of serialization ALWAYS comes back to the user interface. (even if it is a simple box telling the user that there are no rooms left in the hotel due to a transaction failure that caused the room count to drop below zero). You also do not have to assume that the above example of “locking” prevents users from accessing the data. It does not! (that locking example only prevents a process from updating the data..not others from reading it!!! Those process MUST be serailied!) Thus, the issue of ACID in no way address or helps as what you will do when two users try and edit the same word document, or a Excel document or CityDesk document (as in this case)…does it? How does a ACID approach solve the the common problem of two users trying to edit the same word doc? That is the real issue here…not a lack of a transaction based data engine. (since we assume that is a given!!) Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sat 04 Jan | X. J. Scott | 'In general, we do NOT have the technology to update partial parts of a word document. ' Albert, don't you mean to say: 'In particular, we do NOT have the technology to update partial parts of a MS Word document. ' There's lots of programs that support multiple writers on ASCII development documents like .c .cpp .h .txt; I assume that the same could be done for .html documents like Joel uses and could also be done for any proprietary format that the developer wants to provide multiple-writer support for. Sat 04 Jan | harp | Albert, that really is a piss-poor solution. You might not ever encounter it, but your code is buggy. There is nothing to prevent another thread from entering that section of code BEFORE you are able to claim the lock. To ensure that you do indeed own that section of code, you need to ask the operating system for it. Use something along the lines of EnterCriticalSection(). Hope that helps. Sat 04 Jan | SM | harp - did you actually read his post? It is about database record locking across multiple users, it has nothing to do with thread synchronization. Sat 04 Jan | harp | SM, You are supposed to be smart enough to extrapolate. Create a mutex instead of a critical section if you need to work across process boundaries. Encapsulate the locking code within whatever synchronization methodology is required. Hope this helps. Sat 04 Jan | Albert D. Kallal | While my example is in fact for database a lock, there is absolute no difference if it was a request for a thread/process lock either. How do you know what the code looks like that executes that lock? How can anyone possibly make the claim it is possible for two users to execute the code? How do you know what that lock code does? The only thing I have exposed in my example is a method of the tour object called “MyLock” How do you know what the code for MyLock does? I only showed the methoed 'MyLock' MyTour.MyLock You can only guess what that code does for the method MyLock? However, I am not sure why it is poor? I am always open to suggestions. (I leave a few more obvous details out untill this one clears!). Albert D. Kallal Edmonton, Alberta Canada kallal@msn.com Sat 04 Jan | Adam Young | I actually wasn't attacking your solution, BTW. I just wanted to point out that locking was the tip of a much larger iceberg. For instance, if you lock in one application, there is nothing to prevent changes from being made from another application. Same with threading etc. I think Joel's solution may be assuming a single users. It is going against access, which tends to be used for lower end solutions. I doubt he is using global data access objects that can be hit from multiple threads because that is too much writing to do across all these things. More than likely he's done the equivalent of a select for update. Just a couple thoughts. Sat 04 Jan | Albert D. Kallal | >>Albert, that really is a piss-poor solution. You might not ever encounter it, >> Use something along the lines of EnterCriticalSection(). So, what exactly is suppose to happen if EnterCtricial cannot in fact be entered? Hum? What do I do? I certainly don’t want to freeze up the user? On the other hand, how do you know that the code behind my method “MyLock” does not in fact call a EnterCritical routine and ask for a lock? Hum? You don’t? Further, the way my code is setup it is NOT possible for two users to get lock in that example code anyway!. Again, it is moot if we are talking about a record lock, or a process lock. They really are the same thing in the sense that we only allow one user to get that lock, or run that example process. The problem always remains the same: What do you do with the user can’t get that lock, or as in your suggestion one CANNOT Enter “EnterCritical”? Hum, what do you do? You have to tell me? Fact is, we are actually dealing with JET, and that means we are talking about a multi-user system in a file share mode. That means we don’t in fact have a server to manage our program threads…do we now? One of my clients is using that tour software has 6 users in the system all day long. Now, how do I deal the problem of two users trying to grab the last hotel room? (and also NOT having the user get stuck in a dead lock?). By the way the code in “MyLock” does in fact deal with this problem in a very nice way. What would you do? Because really at the end of the day, there is no difference between a user not being able to run your “EnterCritical” and a user trying to run MyLock.. Well, the difference is that I came up with a workable solution…and most people using a file share have not… Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sat 04 Jan | X. J. Scott | Albert, I understood you to mean that you were using Boolean MyObject.MyLock() which was something like: Boolean CriticalObject::MyLock() { mutex_toolkit::EnterCriticalSection(); if (true == this->locked_) return false; else this->locked_ = true; mutex_toolkit::ExitCriticalSection(); return true; } Correct? I assume the detractor did not think you were invoking OS suport for atomic operations within your MyLock(), hence his misunderstanding. Sat 04 Jan | X. J. Scott | Oops, exit the critical section before returning in the first case of the if clause. Sun 05 Jan | harp | Albert, first you said it was a boolean: 'MyObject.MyLock ... The above is a Boolean, and thus' Then it looks like you use it as a boolean: 'If MyObject.MyLock = true then' Now you are saying it is a method: 'I only showed the methoed 'MyLock'... MyTour.MyLock' It being a method would make a slight difference, but only if you wrote some really bad code in that method that accessed some globals. You would still have to use the OS calls to be bug free though. Why do I know this? And how can I possibly make the claim that it is possible for two users to execute the code? Simple. I know what I'm talking about more than roughly 50% of the time. And I've been doing multithreaded windows since it was possible to do multithreaded windows. And I know you develop for windows. And I've made something similar to that mistake before. Whether it's a variable or a method that touches a variable, there's no way possible to guarantee that another section of code doesn't alter that variable (or the critical code in the method) after the other section checks on it, but before it can claim the ownership. Why? Because you have no control over how the OS grants time slices. Take this example: 1: If obj.notlocked 2: obj.lock If you have thread1 and thread2 running through this section of code, there is no gaurantee that just because thread1 got past line 1 successfully that thread2 _the next time slice_ won't make it past line 1 AND line 2. Boom. Deadlock. Sun 05 Jan | harp | Wow XJ, you really took some liberties in assuming what Albert's code did. Are you guys buddies or something? You've also listed a dangerous section of code. You give a naive user of that class the impression that it's thread safe. 1: Boolean CriticalObject::MyLock() 2: { 3: mutex_toolkit::EnterCriticalSection(); 4: if (true == this->locked_) 5: return false; 6: else 7: this->locked_ = true; 8: mutex_toolkit::ExitCriticalSection(); 9: return true; 0: } 2 threads can be queued up on line 4. thread1 executes line 4 and it evaluates false. Then thread2 executes line 4 and guess what? It's still false! Sun 05 Jan | harp | Albert, this is really basic stuff. Have you heard of middleware? Sun 05 Jan | harp | XJ, you caught me on one of my times when I wasn't operating above 50%. Of course mutex_toolkit::EnterCriticalSection() wouldn't allow 2 threads to be in it. Sun 05 Jan | X. J. Scott | Ah good, thanks - saved me from replying. The whole point of the toolkit routine is to ensure that task switching can not occur on a uniprocessor probably by disabling interrupts, or do something more complex likely involving a hardware atomic test and set instruction on shared memory if multiprocessor. Sun 05 Jan | X. J. Scott | 'you really took some liberties in assuming what Albert's code did.' Perhaps. I think that I just assume that if someone is using the word 'lock' that they know what they are doing and the issues involved. I sort of think this is a safe assumption since a naive programmer would be unlikely to be using the word 'lock'. I didn't assume that the code Albert tossed out repersented the actual code but was rather a quick and vague approximation, to be easily filled in by persons familiar with deadlocking and spaghetti eating professors and all the usual suspects. Sun 05 Jan | X. J. Scott | One subtlety to be mentioned is that even if Albert didn't do it right, he's working with a small number of people checking out files in his applications. Overlapping write checkout requests are probably quite rare and the user requests coming in so as to interlace in the right way has probably never happened. If he starts to have more users simultaneously it will be more of a problem. Where I run into these problems is with multiple threads of interacting audio and midi streams where this stuff kicks you in the head instantly and painfully. Sun 05 Jan | Albert D. Kallal | Well, I did state that MyLock is a boolean. I probably should have ALSO stated that MyLock is a Method of my object that returns a Boolean value!! (it is in fact a method of the tour object that returns a value.) Thus, it is some what understandably that I gave the impression that the MyLock is simply a “simple” value (or property) of the object. (and I do allow multi instances of that object by the way). It is a method of the tour object. In addition, it really can be considered a “mutex” for all intensive purposes. It actually tries to do a insert to into a table and looks for a key violation (that key based on the workstation name and a id). So, it is in effect my own mutex (MUtual EXclusion object). In fact, people have used their database system since the dawn of databases as a mutex server! However, this really begs the issue. Even if it is a simple Boolean flag, how in world can you possblity make any assumptions about how the Boolean value got set? It is really is a waste of my time to sit here and justify or even explain that fact that MyLock is a method, or just a property that returns a value. I could sit here and say I am lucky since it is a method that invokes code! However, since none of code showed how that lock value got set I am stumbling to understand why your argument of the fact that it is a simple Boolean value, or a method makes a hill of beans differance? The fact of it being a simple Boolean, or a complex method makes NO DIFFEREANCE!!! Why would it make a difference? I am trying very hard to understand why you are making such wild assumptions about my code? So far we have a assumption that a value is a Boolean and not a method (which, I fail to understand why it makes a difference?). At least you mentioned that method would make a “slight” difference! (it is not big deal!). To qutoe: >>> It being a method would make a slight difference, but only if you wrote some really bad code in that method that accessed some globals. You would still have to use the OS calls to be bug free though. Why do you make the assumption that I have to make a OS call? Why do I have to use the OS calls? You have made zero case here? Do please tell my WHY I have to make a OS call to be bug free? In other words, if I DO NOT make a os call, then I have bugs? Wow!! Why and how can you possibly make such a sweeping statement here? Also, why bring the issue of globules? For what reason do you then need to bring in the issue of globals? For what..to launch another attack on me? Please explain why you would bring up the issue of globals? The person can launch two copies of my software on the same workstation…and it all works fine. In addition, multiple instances of the object often exists. However, again, why bring up the issue me using global variables? (I don’t’, and I can’t anyway). Do you actually think that someone would try and use globals for a process lock? Gee, how stupid do you think the rest of the world is compared to you? It is not even a question of you accusing me of using globals, it is fact of bringing it up, and then stating the ONLY way to be BUG FREE is to use a OS call! In other words, don’t use glboals..and you have to use a OS call (which by the way if funny, since I use neither!!). I am missing something here? I am really a bit confused here! Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sun 05 Jan | harp | Albert, I've already explained all of this above. Which part are you having trouble understanding? Maybe you should give me an example of code that sets a boolean as a lock variable that isn't bug free and doesn't make an OS call. Sun 05 Jan | harp | Oops, _IS_ bug free. Sun 05 Jan | Albert D. Kallal | Well, the problem I am having here is that I am being told that I must make a call to the OS for this process lock? Why? Further, you do realize that with JET we have no central server here. In other words, each user is running their own copy of the code on each pc. There is no server side code running on the server!. In fact, this means that NO code is run on the server, it is ALL on each client computer! However, that above again is really is a moot point. As mentioned, people have been using their database system to create process locks since the dawn of databases that offered record and RI abilities. I see no reason as to why I can’t use the database system in place of a OS call? Why? If you are asking me how JET prevents two people on a network from adding the same key to a table? Well, I guess we would have to delve into the details as to how JET manages a record lock on a network file share. However, I don’t have wait, or figure out how that locking mechanism work, all I have to do is check for a key violation. Either JET allows, or does not allow me to add that key value!! It is one or the other..but I don’t site there waiting). The end result and effect is no different from a process lock. Beyond the above information, anything more is simply an abstraction (ie: I don’t know how EnterCriticalSection() gets its lock, or how JET does. All we both know is that they work and that JET does NOT allow other processes to insert a key that is already in a table.!!). What is the difference? So, my process creates lock by inserting a key value into a table. If that key value is already there..then we get a key volition, and the routine thus does not get a lock! There is certainly no OS call! (of couse..there is no code on the server either…but again that makes no difference). It really is not important, but the code that attempts the lock (lock method of the Tour object eventually calls a general lock routine of mine. The code that we eventually get to is: tblLockRecords!lockitem = strKeyLoc tblLockRecords!NetWorklogon = UserLogonName tblLockRecords!ComputerName = UserMachineName tblLockRecords.Update mylock = True 'lock is succesful Exit Function ErrLock: ' this error assumes that the key was already in the table, and thus the lock ' failed mylock = False tblLockRecords.Requery ' we are still in a edit state...flush/release index lock(s) However, again, I fail to see why the above code makes any difference!. You have not made any case as to why I must make a os call? The above concepts apply 100% to if I was using a server based system. People right now are using MySql on Linux with the inno extensions to act as a Mutex server. Why must I use a OS call here again begs the question? Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sun 05 Jan | harp | JET is doing the locking for you. It is making the OS call. How do you think it prevents 2 simultaneous updates from succeeding? Sun 05 Jan | Bob Greene | Albert, JET handles locking for you. http://www.microsoft.com/ACCESSDEV/Articles/jetlund.htm It looks like you developed your own scheme though. Correct? If so, why? Sun 05 Jan | Albert D. Kallal | Well duh? I mean that is like you claming that I must use assembler to add 2 +2 = 4 I don’t use assembler to add 2 + 2 = 4, but I am sure assembler eventually comes into play. Lets not deal with such a absurd level of semantics here!! Surely you are not grasping at such low straws? I make no calls to the os! Heck, we could be talking about 15 computers in a server based farm, and all 15 of them are connected to a Oracle database.. You are now going to come into the room and suggest to the developers that that 15 computers have to make a os call for a lock? How ridiculous can you get? Further, what really insults me is that you used such a lame argument to state that it is s piss poor solution! That is really low of you! Stating that a computer eventually has to come up with a process lock is fine. I can agree with your statement. However, to blast me publicly with such a lame argument for my approach is where you stooped low. I don’t mind people criticizing me at all. But at least have some foundation to stand on here! Fact is, those 15 computers don’t have to call the OS for a process lock (in fact, it would be stupid if I walked into the room and told the developers they have to!). The standard approach is that the 15 computers requesting the same product, only one will be granted the key value. The rest will fail, and thus they will not be able to run the purchase process code. To say that we solve this problem with a call to the OS has got to be a grasps at some straws. Heck..the straws are so thin I can’t even see them! What a stupid idea! My gosh surely you are not going to stand behind such a lame argument as that? Using the above lame argument to call my approach buggy, and piss poor is not much ride on is it? Gee, I can just see you suggesting to the Oracle developer that you “MUST USE A OS CALL, or else to quote you: “is piss poor approach , or buggy?” Come on here? Really, is that you only answer? That eventually the database system has to do a call to the os for lock? That is like telling me that I have to use a OS call to add 2 + 2. Well, duh!!!…thanks for enlightening us all in this group with the fact that a computer has to run code to accomplish a task! Gee, thanks for that big new piece of info!! To be fair, you were suggesting to make a OS call, and I was most surprised. My code DOES NOT, and DOES NOT HAVE TO make a os call. (sure something down the line might..but that is a silly conjecture) If that is your only argument here, then I have to wonder why my approach is piss poor and buggy? You sure don’t have much to stand on here…do you? Look, at the end of the day, you were assuming that we were taking about a process or thread running on a server. You can simply state that you did not think too much before you make a response! In fact, in most distributed environments, you will wind up using the database server to control these process. I fairly accept your mistaken assumption. However, don’t just sit there and fall over. Simply state, yea… I popped off my top a bit..and should not have stated that my approach is piss poor and buggy. No big deal..I am not looking for a fight here…just a reasoned approach to problem solving. I sincerely believe that you also believe that we are all here to learn more. This is public form..and we all snap a bit. I walk from this one with no bruises or ill feelings. In the scope of things…this is not too important! Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sun 05 Jan | Albert D. Kallal | >>Albert, JET handles locking for you. http://www.microsoft.com/ACCESSDEV/Articles/jetlund.htm It looks like you developed your own scheme though. Correct? If so, why? ================ Good question! I wanted something for controlling processing in addition to record locking. In other words, not only did I need to control some record conflicts, but I also wanted something to control some types of processing. Thus, I need a mechanism for this. Thus, in fact I am really am 'rideing' off of the locking stuff built into the JET system. In other words, I simply try writing out a key value to a table. Either I get the primary key, or I get a error from JET that someone else did!!. Hence, it is really JET that does the dirty work for me and figures out who actually gets the key. Thus, I am using JET as a Mutex (or a Process) server in this case! Also, remember that transactions in JET only allow a transaction to be rolled back, but do not prevent two process from running! (big difference between that). (and course transactions are separate issue from table locking anyway). Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sun 05 Jan | harp | My apologies Albert. I am used to dealing with low level internal details. I sometimes forget that this forum is inhabited by a lot of VB developers. Sun 05 Jan | Albert D. Kallal | Thank you kindly harp. I will say that I am also a bit to blame in running this thread down this path also. I probably should not barked about “why” I can’t lock, and should have really just came out and stated that I am using a database, and how that lock works. It sounds like you think in terms of server based software, or simply in terms of threads (something us VB folks don’t worry about!!). Regardless, the fact that I did have to explain how I accomplish a process lock using a database turns out that this thread does/did have some learning value! Anyway, thank you kindly. Sincerely, Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sun 05 Jan | Herbert Sitz | So how is it that CityDesk would benefit from rolling-your-own locking rather than using the built-in locking offered by Jet? Sun 05 Jan | Adam Young | If I might interpret: Albert, you were asking (amongst other things) Joel how he deals with the processing (UI and/or business) for the case where someone requests a piece of data that has been locked by someone else. In the web, you tend to want Transactions processed completely between web page requests. The main reson is that otherwise you will lock any resources in early requests until the transaction completes or rolls back. Since we are dealing with the web, this is possibly going to be when the session times out. If your session time out is on the nature of 10 minutes (not unusual) you will have lots of locked resources. So with the 'Reserve the last hotel room' problem you spoke about before, you are going to have a race condition. Who ever gets there first wins. You may see in one vioew that the room is avaialbe. Next view it may not be. You have to check at the Reserve event/request whether or not it is still available. For the Content Management issue, it is more significant if two people are updating a particular piece of content at the same time, as there is no good way to merge changes, and the web does not allow you to save locally. At least not easily. I think in this case, you don't just want to depend on the DB, but want an state field on the row in question, that gets updated. This can help out workflow, since instead of just locked and unlocked you can have new, locked for editing (with locked byu another field in the table), approved, posted, removed and what ever other states you want. Yes, you have to be sure to unlock at session time out, but you can also allow a senior member of the team to grab the lock : John has an emergency and will be gone for two weeks. Everything he has been working on gets reassigned to a co-worker by hi boss. You get the idea. BTW, something like CVS may be a better tool for the content storage facility than a DB. You can have an article ID stored in the DB, and the actual article stored in the file system, with a tie in to a revision control system. Again, just a couple thoughts. Data Access code and Refactoring | Fri 03 Jan | Richard Caetano Joel, I was curious about your post. Do you find that most of the time you write code that gets the job done, but then later come back and refactor which result in nice objects like the entity objects that you mentioned? I find myself doing that although theres a little code angel that sits on my shoulder telling me to write the entity objects first. I just find it more applicable to write code that solves a problem and then come back and make it pretty. The solve problem->refactor coding cycle works well for me. Its sorta like the make it work then make it perform approach. Ideas or thoughts? Fri 03 Jan | Joel Spolsky | In almost every large project I've worked on, I've done just what you described. Both CityDesk and FogBUGZ started out as big hairballs. FogBUGZ got massively refactored last year (I wrote an article about it here somewhere); CityDesk is in much better shape but will undergo the same thing this year. Sat 04 Jan | anonQAguy | You gain a lot greater understanding of the problem involved by plowing through it once first, even if the solution might be messy and/or fragile. The first attempt, focused primarily at just basically solving the problem serves almost as a working prototype one of whose primary purposes is to be as much a tool for analysis and requirements refinement as it is to satisfy the end objective. This works in other fields of course, too. When working for an analytical agency some years ago doing Operations Research work, one of our chief analysts formalized what they called a 'Model-Test-Model' approach which was basically the thing you're talking about. There the 'model' they referred to was usually one of our force-on-force combat computer simulations. They'd do a simulation run (well, often a set of runs), get some answers, then refine the model or the parameters to clean up problems they found or to reflect understanding they gained, then more runs, then refine, etc. This helped especially for problems that were particularly complex or somewhat nebulous for some reason or other (like Joe's 'hairballs'), where maybe they needed to learn more about what questions they needed to be delving into. As Sarek of Vulcan said on one of the ST movies 'It is hard to answer when one does not understand the question', or words to that effect. M-T-M was not as necessary or efficient when the problems could be better defined initially and the particular phenomena were better understood--equivalent to when you're developing a smaller, simpler, or more familiar application where requirements analysis and design could be much more complete right out of the gate. For my money, this points out why there are such problems with trying to stick to pure waterfall for big, complex, or ill-defined projects--you can only guestimate so much about the problems when all you have are static abstractions of the system. In fact, I wonder if anybody really uses pure waterfall anymore for anything but smaller, very clear-cut problems? Cheers, Sat 04 Jan | anonQAguy | Oh - and from the perspective of testing, I generally follow a similar pattern. Of course it helps that our shop works iteratively on pretty much everything, because that tends to naturally reinforce me using this approach. In general, though, I'll normally start off with plowing thorough the main workflows, following the 'happy paths', seeking to understand the system and watching for branches in the flows and potential error conditions I'll want to come back and add tests for later. This is especially true when we haven't been able to do as much up front analysis as we would have liked, and therefore I haven't been able to do as much preliminary test design as I would have liked. Sat 04 Jan | John Topley | The article Joel wrote is here: http://www.joelonsoftware.com/news/fog0000000328.html Sat 04 Jan | T. Norman | I find that it is a good approach if you have the opportunity to do both the barely-working way and the clean neat way before releasing it to production. But if you don't have that luxury, I think it is better to do it clean the first time, instead of having someone else other than the original developer picking up the code 1 year later to try to figure out the old hodge-podge and risk breaking other code that was since built on top of it. Sat 04 Jan | Adam Young | THing is, after you two or three porjects, you realize that you are solving the same problems over and over. That is how frameworks develop. In the web for data entry it is almost always, search, fetch values, display values, get input from form, validate, store or report error. The Form is in a client area of the screen, with navigation information around the edges. You drill down into information heirarchically. The parts of this system that should be easily config/findable are: The values on a give form and the drilldown heirarchy. I think I would map out how I was going to to solve these problems in the abstract before tackling another one of the type of projects. Sat 04 Jan | X. J. Scott | I'm with Richard and Joel. I've tried it both ways and the way to do it is to go for the big ball of mud design pattern first and then refactor it. A good time to refactor a section is of course just before trying to add something to already working code. The problem in architecting something that has never been done before is that until you have implemented it, you don't really know where the problems are. Come up with your mud ball first and see where the med is especially lumpy. Years ago we realized that 'premature optimization is the root of all evil'. This is because we can not predict where the actual bottlenecks will be until we are finished and run the experiment of doing profiling. Everyone knows this is true now and no one reasonable optimizes before its time because you end up messing with the wrong stuff. Today, we have a similar situation but on a higher level. Here is my new wisdom for the modern era: 'Premature Architecture is the root of all evil.' -- X. J. Scott Sat 04 Jan | Adam Young | BTW, I am a huge proponent of refactoring. Fowler should get an honorary degree for what he did with that book. Yes, I know it was the subject of his Doctoral work. However, many a project is in desperate need of refactoring , with out enough time alloted to do it. Many a project suffers from Architectural Arthritus (I made that term up, but feel free to use it) that you just have to deal with it. And sometimes, you have already solved a problem, so there is no use in avoiding it just to get right into the solution. Personally, what I like to do is to map out the objects/entities of a project right at the start, and then use that map as a guide. It serves the role of the common Metaphor from XP. Sat 04 Jan | PC | When I was a graduate student I once took a course in how to write papers. The idea was to start out with rough drafts and outlines until you get a general idea of what you want to say, and then just keep revising and re-writing until the paper is good enough (or you run out of time). When I started programming I applied the ideas I had learned about writing papers (start with rough drafts and outlines, revise and revise). I assumed I was doing it all wrong, but I had learned to like working that way. Then I found out I was refactoring, and now it's considered a reasonable way to program. Before graduate school I used to hate revising, and wrote papers once from beginning to end. Maybe refactoring, like revising, is a practice you have to learn to appreciate. Sat 04 Jan | Leonardo Herrera | My own experience with architecture/refactoring was in a dot com company. I designed a very nice set of objects that mapped a lot of common operations in a rather large website. It was cool, a lot of functionality got created in short development cycles. But then, something strange happened: the mission of the company actually changed two or three times in a short amount of time, so the web app had to change accordingly. As a result, the programmers in this project started doing workarounds to solve the lack of functionality (instead of refactoring the objects), and in the end we got our own hairball, complete with strange wiring around the initial entities. Now I'm minimalist :-) Sun 05 Jan | anon | I believe this is similar to 'bottom-up programming.' So...Joel's Still Using DAO.  Is Anyone Else? | Fri 03 Jan | Norrick Reading Joels latest post on the front page, Im pleasantly surprised to find that CityDesk uses DAO. Its a nice example of if it aint broke, dont fix it - DAO remains perfectly adequate for use in desktop applications, despite the push to move to ADO for everything. Now Im curious - is anyone else releasing apps into the wild that use DAO? And is DAO even going to be supportable in future versions of Windows? Fri 03 Jan | Norrick | P.S. Thanks for the documentation, Joel...I can use that idea on a project I have coming up. Which makes me wonder...I don't suppose you'd be willing to save me a lot of time and share the source code to your entity classes, wouldja? ;) Heh. Fri 03 Jan | HeyMacarana | Does anyone use or can recommend a good data layer generator? I.e. something that will take as input a database of tables and generate a default general data access layer code for you that can be customized later on through overrides, etc.? Fri 03 Jan | Daniel Shchyokin | If youre using access why not? Fri 03 Jan | anonymous | This is the first time I've heard the term 'Entity Class' used to describe the process of creating a class for each table/entity in the database. I always thought it was common practice. I've been using this technique since I was in school. heh. And yes, (before I lost my job) I used DAO and ADO every day. Fri 03 Jan | Joel Spolsky | I made up the term 'entity,' it is fairly common practice but I don't know what other people call it :) When you're coding in MFC with DAO there's some kind of class wizard that sort-of autogenerates these things for you, but the autogenerated stuff is never as good as what you would create using just-in-time class design. (When we create entity classes we start with virtually nothing, and only add methods as we discover a need for them. Sometimes you don't have any List method, sometimes you have 12 varieties (e.g. ListNonDeleted, ListInFolder, ListByDate, etc.) Since our code all maps directly to our tables, there's not much point in sharing them, but it might be a neat idea to ship the entity classes, as source code, to people who want to write add-ins or extensions to CityDesk. Fri 03 Jan | Xxxxx Xxx | If you want to target Windows95 clients then ADO is not feasible. You absolutely won't want user to download huge ADO runtime. Fri 03 Jan | Philip Dickerson | In response to the question about a data layer generator: This book has a concept of 'Data Objects' (in VB) that are closely tied to database tables: http://www.amazon.com/exec/obidos/tg/detail/-/1861002580 Enterprise Application Architecture with VB, ASP and MTS by Joseph Moniz, Wrox Press Inc; ISBN: 1861002580; 1st edition (May 1999) I read it a few years ago, and I don't totally agree with the approach described in the book, but the data object portion may be relevant for this topic. The book author also wrote a VB application - 'Object Factory' - that is intended to generate all the VB data access code as well as the database tables by asking you to enter essentially names and datatypes for all the properties of the object (which then also become columns in the corresponding database table) The 'Object Factory' code (and other sample code) is available for download at: http://www.wrox.com/dynamic/books/download.aspx?isbn=1861002580 The download requires registration. Sat 04 Jan | Chris Tavares | It's not quite an 'entity generator' but if you're working with .NET then you might want to check out Deklarit at http://www.deklarit.com. It takes as input a description of your 'entities' and generates not only C# code, but a normalized database schema as well. I found it to be a great help for the relationally challenged (myself included). -Chris P.S. I did get paid to write a white paper for Deklarit, but I still think it's a cool product. Sat 04 Jan | Troy King | Joel > I made up the term 'entity,' it is fairly common practice but I don't know what other people call it You just happened to make up the same term that just about everyone else calls it. In relational db theory, entities are usually implemented as tables, so referring to your table wrapper as an entity is pretty common. Sat 04 Jan | Philippe Back | For a good generator, supporting both .NET and VS6, have a look at www.pragmatier.com You can download a version using MS Access for free. It generates all you need based on class descriptions and you can also retrofit exisiting tables in it. Sat 04 Jan | ODN | For a good discussion of the concepts behind mapping to relational databases, check out the sample chapter from Martin Fowler's new book, Patterns of Enterprise Application Architecture. The chapter is titled 'Mapping to Relational Databases'. http://www.aw.com/samplechapter/0321127420.pdf If you like this stuff, you can still get old versions of the rest of the book's content at http://web.archive.org/web/20011110053223/http://www.martinfowler.com/isa/ Another good read for those who use .NET is the Microsoft whitepaper 'Designing Data Tier components and Passing Data Through Tiers' at http://msdn.microsoft.com/architecture/default.aspx?pull=/library/en-us/dnbda/html/BOAGag.asp It covers the pros and cons of 5 different approaches to implementing business entities. Sat 04 Jan | Nick Hebb | Back to the original question - yes, I still use DAO when working with Access. From what I've read, it benchmarks faster than ADO when used with Access, and it has a much richer feature set. More importantly, though, sticking with DAO for VB <--> Access work makes the code more consistent through time. Sat 04 Jan | Albert D. Kallal | Well, if you are using JET you do have DAO installed. DAO is included, and is part of JET. You can use ADO, but if you are talking about a standalone application, then ADO just sites on top of JET/DAO anyway. In other words, you really can not avoid DAO when using access/mdb file format. (ie: you might not use it…but is going to be on your computer!!). In addition ADO does NOT have a data engine,but can only REQUEST data from some data engine. Hence, if you are writing a standalone, or one that will not involve a server, then it makes no sense to use ADO. However, ADO is one of those leaky abstractions that Joel often talks about. This means that all your code will still work if you dump JET and change the data engine to SQL-server. So, for sure there are advantages to use ADO. ADO is simply is another layer of abstraction that is put on top of your database engine. (be it JET, or SQL server). Using it allows you to “dump” or change the data engine. (so, I can dump Microsoft’s engine and use MySql….see..it may not be such a brilliant idea to have told all us developers to dump DAO/JET!!!). Microsoft should be happy that many of use still use DAO. Since, if I had adopted ADO…then I can change data engine makers a lot easier! (Hence, too much abstraction can allow your customers to change!!) If your application is un-lkey to need a server, then I would not use ADO unless that is all you know. Also, DAO has more functionality, and exposes a lot more of the JET object model. Of couse..it also performs better (but not too big of a deal these days). We have been told since what? 1998?, that JET is gone, and not to be used? Well, I hate to say it..but it is still alive and well. So, yes..I do recommend that one learn and use ADO..but there is NOTHING wrong with writing and using DAO. In fact, for a JET standalone app..it is still a better choice, and I still use DAO. Also, we don't know that Joel is using DAO in his code, he could be using ADO? (or is there some ref I don't know about?). Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sat 04 Jan | Albert D. Kallal | >> Also, we don't know that Joel is using DAO in his code, he could be using ADO? (or is there some ref I don't know about?). Oh..ok just look at the front page now!!! duh!!! Sat 04 Jan | Adam Young | Seems to me that Code Generation in Visual C++ could be based on DAO or on straight ODBC. If you are already performing all data access inside C++ Code, why would you go to DAO and not ODBC? The Class (Do they call it result set?) that wrapps it already provides most of what you would get from wrapping the entities, plus there is a method (no code generator in VC6 which was the last I used) to modify the result set to do bulk loading, which was much more effecient. I can't beleive how much I just pulled out of long term memory. Sun 05 Jan | HeyMacarana | Thanks for the entity generator links guys! Very interesting stuff. I'll have to download and try out the programs plus check out the articles. Sun 05 Jan | Jonah Burke | Joel, you made up the term 'entity'? Are you sure you didn't steal it from J2EE? Sun 05 Jan | Joel Spolsky | i borrowed it from ERWIN and the whole notion of entity-relation diagrams, in which the term 'entity' refers to a logical thing corresponding to something in the real world, often corresponding to a table but not always. Sun 05 Jan | braid_ged | A large Java software project I worked on a year ago had an 'Entity Layer'. Which was exactly a group of classes each relating to a database table and a few classes relating to multiple tables. I have been assuming this term was indistry standard, obviously it's not as ubiquitous as I thought, given we are debating where it came from. If proper relational database design instructs you to create tables based on the 'entities' you are storing data about, and modern OO programming design instructs you to create classes based on what 'entities' are present in the system, this type of entity layer is almost a fate acomplee. The project by the way, crashed and burned, partially due to badly desinged entity classes that made it difficult to actually encode the required business logic and certainly broke Joel's rule about not compromising database efficiency. Issues with Entities that have created Problems : - will entities need to be nested ? if so what goes inside what ? - do you want array of entity objects being passed around the place or do you want 'super entities' that hold all the related entities. - if you need information on entity X simply to instantiate entity Y do you go through the entity to get at it or make entity Y able to instatiate itself based on this entity x data. - how much of an entity should be loaded from the database at different times ? - how to manage multiple entities all instantiated all in different stages of development at the same point of progress through the business process. Were you to commit multiple entities to the database would result in inconsistencies between them. Yet an entity is waiting for something before it can proceed. These are badly written up points, too vague to really be analysed. I will think about it all some more and post again. I really need to figure out why our entity classes were so bad. I guess this is the heart of computer programming, abstracting and breaking hard problems in smaller blocks, I suppose one major problem was simply the lack of experience of the programmers. Braid_ged. Making it on your own | Fri 03 Jan | John Mohsen Id like to hear about experiences in starting your own business, software related or otherwise, or links to such experiences (besides the story of fog creek since theres this lil website that mostly covers it). Tell us how you got your first customers/clients, or when you actually knew you were doing alright and were going to make it as a consultant/entreprenuer, the uncertainty, the blood and sweat involved (or not), successes, failures, etc. Not particularly interested in rags-to-riches, but rather in those making a decent living. Fri 03 Jan | Matthew Lock | Paul Graham who developed Yahoo store with some others has great articles on starting a software company: The Other Road Ahead http://www.paulgraham.com/road.html Beating the Averages http://www.paulgraham.com/paulgraham/avg.html Revenge of the Nerds http://www.paulgraham.com/icad.html Also this site 'Joel on Software' is hard to beat: See the 'Software Strategy: The Big Picture ' section: http://www.joelonsoftware.com/navLinks/fog0000000247.html A great book too is: 'Hard Drive: Bill Gates and the Making of the Microsoft Empire' http://www.amazon.com/exec/obidos/ASIN/0887306292/qid=1041588787/sr=2-1/ref=sr_2_1/103-8965068-7643832 Fri 03 Jan | Matt H. | I'm currently reading an old edition of 'Million Dollar Consulting' that I found on our book-shelf one day: http://www.amazon.com/exec/obidos/tg/detail/-/007138703X/qid=1041598516/sr=1-1/ref=sr_1_1/103-3829423-9530239?v=glance&s=books It's REALLY GOOD. A few key ideas: 1) If you want to make big bucks, stop charging by the hour and start charging by the project. (People balk when they hear you charge $200/hour, but if they just hear that a perl script costs $1,600 to develop, they might just buy it. You don't have to tell them that you can write and test it in eight hours ...) 2) You can choose to compete in products, services, or relationships. If you try to be world-class in all three, you'll kill yourself. He suggests being good at products, better at services, world-class at relationships. (Very similar to what Joel has taken: All his community-building stuff is world-class relationships.) 3) If you want to consult, start with: A) Calling local non-profits and governments and doing pro-bono work, B) Bidding on government contract work, C) Make sure your name appears on the resource list for governments, non-profits, and large businesses. 4) Articles/Presentations: Get your name out there. (Think Joel, AGAIN ...) 5) How much you need to spend to get a professional office, answering service (not machine!), FAX, photocopier, Logo, etc. It's about $10,000. Spend less, and you look like a Lone Wolf. Spend more and create debt ... 6) Which leads to: How do you finance your consulting shop? Tons of stuff on strategy and growing your business. I highly recommend it. You can probably get it for free from a library in your area. regards, Matt H. Fri 03 Jan | Philippe Back | Yes, Million Dollar Consulting is a good resource. It's good advice and at the same time, it's not so simple to have everything up and running like he says... It's forward moving tough. Fri 03 Jan | Stephen Jones | Matt, Pretty neat post on consulting. I'm stumped by the $10,000 though. If you're hiring an office for yourself it's too low, unless you're out in the sticks, and if it's just for equipment then it's way too high. You can get a fax number free if you pay $10 or so a month for outgoing faxes, and the faxes will be emailed to your computer. Much better than having a physical fax machine or tying up a whole phone line simply to be able to receive faxes on the computer. You could pay an agency to take your calls so it sounds as if you have a dedicated secretary. I'm not sure you would not be better off simply giving your mobile number (appeaiing lean might have advantages in today's market). Probably the best thing to do is to get the agency secretary to give out your mobile number when somebocy calls in. Fri 03 Jan |     | > You can get a fax number free if you pay $10 or so a month for outgoing faxes, and the faxes will be emailed to your computer. Much better than having a physical fax machine or tying up a whole phone line simply to be able to receive faxes on the computer. < You can setup a faxserver for the cost of a computer...$200 bucks and charge people for faxes. If anyone is interested I'll do it and you can fax through my system Fri 03 Jan | w.h. | Be very careful about fixed-price contracts, especially if you are starting out. If you charge $1000 and figure that it's 5 hours at $200/hr, you had better be prepared for the case where it takes 10 hours and you only make $100/hr because you overestimated it. I've seen quite a few companies end up shooting themselves in the foot that way. My strategy was a very good estimate for the initial project cost and an hourly fee, and then let the client do change requests and go over the initial estimate. Sat 04 Jan | consultant | I've almost lost my shirt on the fixed price contracts I have done. You might be able to get away with a $1600 perl script, but those types of contracts are few and far between. Most single person development projects the client is looking to spend between $20K and $50K, and if you charge fixed $20K, you are going to eat the $30K yourself, when the client squeezes you with feature creep. Sat 04 Jan | T. Norman | Feature creep is a problem only if your management sucks and doesn't know how to track requirements. In a fixed-price project you charge the client for every additional feature. In fact, that's how some contractors make money despite bidding low in the beginning. They lowball the competition to win the contract, because they know the project is one that is likely to require many modifications, then as the customer wants more and more they tack it on to the bill, and in the end the changes add up enough that they make a decent profit. The main problem with fixed-price projects is creating a good estimate that won't put you in the red. It is especially difficult with software, as managers almost always grossly underestimate the cost and effort of software projects. Sat 04 Jan | consultant | >>Feature creep is a problem only if your management sucks and doesn't know how to track requirements. In a fixed-price project you charge the client for every additional feature I agree with this, but getting fixed price to work as a one-man shop, just starting out, is VERY hard. First of all, the client will not want to pay for a design phase, where you cook up the requirements. Thus, it is hard to get good requirements whilst getting paid, in the first place. The requirements will change constantly throughout the development of the software. If someone decides that they want their workflow app to flow in a different direction, after the first rev, they are usually _not_ happy when you say 'that will be an additional $8000.' However, if it is priced by the hour, you just continue getting paid your hourly rate. I'm not saying it is impossible to do fixed price, it is just a lot harder to do than hourly, _especially_ if you are just starting out. I question a consulting book that recommends fixed price to a beginning consultant, since nearly all other resources caution against fixed price contracts. Sun 05 Jan | Phibian | 'In fact, that's how some contractors make money despite bidding low in the beginning. They lowball the competition to win the contract, because they know the project is one that is likely to require many modifications, then as the customer wants more and more they tack it on to the bill, and in the end the changes add up enough that they make a decent profit.' Should note that this is generally referred to 'bait and switch' by the client, and is not conducive to repeat business nor recommendations... (So if you are counting on word of mouth to spread the news about you, or if the client has the potential to send a lot of business your way, I wouldn't recommend setting up a lowball bid based on the expectation that 'add-ons' will make up the difference). There's always a delicate balance between avoiding the perception of bait and switch while preventing the client from bankrupting the project via feature creep. The best way to handle this is to have a very clear set of 'included' features from the get go and make sure the client understands that changes from this feature set are billable. Also, it is better to defer many feature changes until a 'Phase II' of the project - since they are often nice-to-haves as opposed to critical requirements (provided your needs analysis was appropriately done), and since it will be difficult to stay within your original time frame if you add many new features (as opposed to substitutions). Of course, that doesn't mean being completely inflexible / insensitive to what the client wants ... best way to learn SQL? | Thu 02 Jan | tim I graduate in May and a friend of mine is going to try to help me get a job w/ his company (which is nice since 6 out of 33 Dec graduates in cs at my school got job offers). The thing is, I need to learn SQL and preferably .Net, javascript and/or vbscript. Anyways, what is the best way to go about learning SQL? The computers I have at my disposal are a pc (1 GHz athlon w/ 512 ram, running WinXP Pro) and a laptop (1.2GHz w/ 256 mb ram running WinXP Home). Thanks to school connections (MS license for the cs department = burn as much MS onto cds as you have cds for) I have access to all MS software. Whats the best way to go about learning sql? Im lookin at the w3schools website, but are there any books you would recommend? Any software? Thanks for your time. Thu 02 Jan | Matthew Lock | Do you mean the actual SQL language, or MS SQL Server? Thu 02 Jan | John C. | If you're on a budget and need to learn a variety of topics over the next few months, I heartily recommend O'Reilly's Safari service at safari.oreilly.com. For $10 a month you get 5 points that can be applied to get the full contents of technical books online. For the most part 1 book = 1 point, and the selection is not limited to O'Reilly titles. You can change titles out once a month. I can't recommend any specific books on the topics you're interested in, but one general piece of advice I'd offer is this: Pick a project that you find personally interesting that would take advantage of all these technologies, and then go build it. You'll learn more from doing real-world problem-solving, IMHO, than simply walking through prefab exercises in books. A natural project given the specific areas you want to learn would be to build a database-driven Web site. Thu 02 Jan | Herbert Sitz | SQL for Dummies is actually not bad as an introduction. SQL for Mere Mortals (I think that's it, in any case the SQL companion to Databases for Mere Mortals) is highly regarded as an introduction, though I've never read it. Using the visual Query Design View in MS Access can be a good way to see how things work, especially if your mind works visually. You can toggle back and forth between the visual representatiion and the SQL that the design generates. Or you can write the SQL yourself and let Access generate the visual representation. Access/Jet SQL has a few peculiarities, but you won't likely run across those until you get to more advanced queries. Thu 02 Jan | Eric Debois | One way to go at it is to use MS access where you can build queries with a graphical interface or with SQL, and switch back and forth between to two views. Access SQL isnt the easiest to decipher with lots of redundant [] and stuff. But it can interpret regular sql well enough so its a good interface for practising. But imho, there is no point in learning SQL (beyond the basics anyway) before you are confident with relational data modelling. Thu 02 Jan | tim | To Matt: I am wanting to learn the SQL language, although my friend's emloyere is a 95% ms based shop. To Herb: Thanks for the book recommendations. To Eric: Any book (or website) recommendations for relational databases? Thanks guys. Thu 02 Jan | Jeff | Take a look at SQL for Web Nerds by Philip Greenspun at http://philip.greenspun.com/sql/ It uses Oracle as its database, but the SQL part is generic. Also, there is A Gentle Introduction to SQL at http://sqlzoo.net/ Thu 02 Jan |     | Leanring basic SQL will take you as long as leanring HTML. About a day. If you know what a database is and how it is structured you shouldn't have a problem. I am surprised you have a CS and don't know SQL; it is a simple tool...do you know what grep it? For stored procedures PLSQL or TransactSQL is in order but you can use google for that. Thu 02 Jan | soda | SQL is very easy, as it was designed to be a language for business people. like someone else said, it should take about a day. that said, it is worth your time to actually take a day to learn it well enough so that you can use it properly. i have been repeatedly horrified by the 'recent CS grad' who thinks it is appropriate to only learn 'SELECT * FROM FOO' and then use loops in the calling language to pull the desired information out of the result set. the greenspun reference mentioned is ok, and joe celko's 'SQL for Smarties' is pretty good for the concepts and code, but not written very well. Fri 03 Jan | Herbert Sitz | Sorry, but saying 'SQL is easy you can learn it in a day' reminds me of the thread currently going where web programmers are complaining that people think they're just web page designers and their 12 year olds can give some tips. Well, I'm a database programmer and I get tired of people saying, 'Oh, yeah, I know SQL.' (No offense, Soda, I have no idea whether you're one of these people or not. But I do know there are a lot of them out there.) Yes, it is simple to learn the basics of the SELECT query. But there are way too many little things you've got to pick up and pay attention to for SQL to be a quick learn. Difference between inner and outer joins is a biggie that you likely won't get the feel for until you've worked with things a bit. Likewise dealing with NULLs and how they can foul things up. Issues with queries that are updateable vs. non-updateable. Optimizing queries. How to deal with parameters in your queries. Then start doing grouping and problems get multiplied. I could go on, and we haven't even gotten to stored procedures and related stuff. One big problem with people who jump into SQL and think it's easy is that their queries often are wrong. That is, they write a query that seems to return the data they want and they think they've got it. But in fact their result set includes some records that weren't really wanted or some records are left out that the programmer actuall wanted to include. And the 'SQL is easy' programmer never notices. It's often easy to write something that gets you close. But to be confident you're getting exactly the right result set in a more complicated query is not something you can pick up that quickly. Also, Celko's SQL for Smarties book would be a miserable introduction to SQL. (A clue: the subtitle of the book is: Advanced SQL Programming.) It's not a systematic treatment of SQL at all. Per Celko himself, '[SQL for Smarties] is for the working SQL programmer who wants to pick up some advanced programming tips and techniques. IT ASSUMES THAT THE READER IS AN SQL PROGRAMMER WITH A YEAR OF ACTUAL EXPERIENCE.' (from the Introduction) What's needed for the original poster in this thread are some introductory books and some playing around with data and queries. SQL for Dummies really isn't for dummies at all. Someone remarked that some reading on database design is also necessary. I agree. SQL for Dummies has a decent introduction to normalization issues, what normalization is and why it's necessary. There's also a pretty good whitepaper on normalization on MSDN someone. I'll try to post the link if I can find it. Fri 03 Jan | Matthew Lock | The best way to learn SQL, or any computer language, is the same as the way to learn a human language - immerse yourself in it. Do nothing for a week but practice SQL queries. Install MySQL on your machine and build a database directly through the command line using just SQL. Populate the database and practice doing queries on it. Once you have mastered the subset of SQL that MySQL supports install PostgreSQL and learn about views and subselects etc. Fri 03 Jan | Jan Derk | SQL for Mere Mortals [..] is highly regarded as an introduction, though I've never read it. I did read SQL for Mere Mortals, but would not recommend it. The problem is that it only covers SELECT queries (and doing a fine job at that). There's not one single example of inserting or updating data. My guess is that it is targeted at managers not programmers. Fri 03 Jan | Mathematical Dunce | Herbert talks a lot of sense. I would also advise that before trying to do too much with the language you make sure that you understand the relational algebra underlying it. I've seen several people with excellent SQL language knowledge beating their brains out over 'wrong' results because they are failing to consider the basic facts about sets and set operations, or are imposing baseless presuppositions about the order in which things will happen. Don't think it will be too hard or too mathematical - if I can understand it, it can't be. An excellent distance-learning course on this is run by the UK's Open University, but you don't have that sort of time. As far as books go, surely C.J.Date 'An Introduction to Database Systems' is a good grounding in all the essentials. Fri 03 Jan | | Herbert Sitz, Nice post!!!!!!!!!!!! Sometimes it seems as if all the online forums are filled with newbies trying to advise other newbies. Just about everything related to software development seems easy until you try it yourself and then your work gets scrutinized by other developers. Fri 03 Jan | Albert D. Kallal | SQL is perhaps the best investment of ANY ONE skill you can learn. I started using SQL more than 10 years ago. I still use it today. Virtually EVERY database system in the world uses SQL. I used SQL on TONS of systems. In fact, in the last 10 years, it is the ONLY language I have learned that I still use. (everthing else I have had learn over and over) Learn SQL today, and you will still use it 20 years from now. Every single business application today will use sql. It is that simple. Now, as for those who state SQL is easy? Ok, lets try two my interview questions on you: I have a table of car salesman. I also have a child table that lists all the cars that each salesman sold. Hence, we have a classic and standard one to many relation between the salesman, and the table of cars that they sold. Ok, with this classic parent to child relation, write me a SQL query that shows each sales man that sold a blue and a red car? Thus, I want you to list the salesman name, and the color,make, model of the cars they sold. Thus: tblSalesMan ID, SalesRepName tblCars Sales_id, Color, Make, Model, SalesDate I will be nice, and REPEAT THE question (since 9 out of 10 people get this wrong): I want a sales man that sold red * A N D * a blue car. (they can have sold lots of cars of each color, THIS IS NOT A TRICK question). Note of course the above question is classic, since it is same type of question like: what customers had a invoice in the last two months? Or what customer bought a product in each of the last two years? This is extremely common question. Anyway…anyone want to post a easy solution? Problem #2 Give me a list of salesman and the last car they sold (SalesRep, color, make, model, salesdate). Thus, I only want a nice listing of the salesman, and the make, model, color etc of the LAST CAR that they sold. Again, a VERY COMMON question (ie: when was the last trip, last purchase, last doctor visit…again VERY VERY common). Ok, folks, the two simplest and everyday questions Lets have all of those who said SQL is real easy post me an answer to the above incredible EVERY DAY questions that virtually every busines I know asks? Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Fri 03 Jan | ODN | I believe the MSDN whitepaper on normalization that Herbert mentioned can be found at http://msdn.microsoft.com/library/en-us/dnacc2k2/html/odc_FMSNormalization.asp It says it 'applies to Microsoft Access', but the advice is relevant to most any database. Fri 03 Jan | tim | Thanks a lot for the posts guys. Feel free to keep posting. I'm currently reading the 'SQL for web nerds' site to get a basic understanding. I'll probably also figure out what class at ISU teaches SQL (there is one but I took a precursor to it before they modified the program) and look at any lecture notes that are online. Anyways, thanks again. Fri 03 Jan | tapiwa | Albert, there is a small problem with your data model. You need to have a salesman's ID in the tblCars as a foreign key. #1. Select t1.ID, t3.SalesRepName from tblcars as t1., tblcars as t2 where t1.ID = t2.ID and t1.ID = t3.ID and t1.Color = 'red' and t2.Color = 'blue' group by t1.ID; Fri 03 Jan | apw | try 'The Practical SQL Handbook' Fri 03 Jan | Herbert Sitz | Tapiwa -- You need to specify the t3 table, of course. But I like that solution. Clever. I think I usually do that sort of thing with subselects; I like your solution better. Fri 03 Jan | Eric Debois | I would probably do I like this, but I bet there is something wrong with this approach... SELECT tblSalesman.Sales_rep_Name FROM tblCars AS t1, tblCars AS t2 INNER JOIN tblSalesman ON t1.ID = tblSalesman.ID WHERE t1.color = 'RED' AND t2.color = 'BLUE' GROUP BY tblSalesman.Sales_rep_Name; Fri 03 Jan | Herbert Sitz | Eric - Yes, you only have one join for three tables. You could do it like that, but you'd need to have two INNER JOINS in the FROM clause (and I think there'll be a syntax error in what you've got now because the join of t2 references t1 and tblSalesman, not t2 at all). Once you do that it's going to be not quite as concise as tapiwa's version, though maybe more explicit about what's going on. Fri 03 Jan | Nathan | i would do it this way: select repname from tblsalesman join tblcars on tblsalesman.id = tblcars.sales_id where repname in (select repname from tblsalesman join tblcars on tblsalesman.ID = tblcars.sales_id where tblcars.color = 'Red') and tblcars.color = 'Blue' i guess it's not elegant, but it works. Fri 03 Jan | soda | Herbert, I stand by my claim that SQL is easy. I no longer do much database programming, but at one point in my life I was the database guy, and regretfully have about 5 years of Oracle experience. I was responsible for training new recruits at a defunct startup, and we took people from zero through outer/inner joins, subselects, grouping, aggregates, transactions, trees, and intro PL/SQL in about five days. The fact is you can learn inserts, deletes, transactions, selects, subselects, inner/outer joins in about 1 day, and that is enough to do 80% of what most people ever need to do. I recommended Celko's book because every intro book I have seen is 10% useful and 90% garbage. I don't like the Celko book because it is poorly written, but there isn't any other book with the same content. You can learn the easy stuff with no book and the Celko book is really the only other thing out there. If you are using Oracle (which didn't seem to be the case in the original posting), most of the Oracle press books are reasonably good. Fri 03 Jan | Herbert Sitz | Soda -- That's fair. I still stand by what I said, though. Go ahead and lead some guys through SQL in a quick crash course. They'll even understand it and see how it works. Then set them loose on their own with a real-world database. They'll be able to do simple queries, sure. But I wouldn't trust them to get the correct results on anything very complicated. They're going to have to suffer through a learning curve. Not a terribly steep one or long one, as far as programming-related topics go. But steep and long enough that I think it's very misleading to say 'SQL is easy you can learn it in a day.' And if they're trying to integrate SQL into programs they're going to have to deal with lots more issues. How to embed SQL in their programs. Assembly of SQL strings, proper quoting or nonquoting of variables, etc. Yes, you can show them this stuff in a day, but it's going to take longer and they're going to have to suffer through frustration and mistakes before they become proficient, and more importantly, before they can reliably create complex queries. On a more general level, I think the number of programming-related things you can learn in a day is vastly overestimated. You can pick just about any topic and find someone giving a 1 to 5 day training seminar on it. Can you say someone who attends that seminar 'knows' the subject? In almost every case I'd say, 'No.' You only learn by doing it on your own and suffering through your own frustration and mistakes. Sure an introduction and overview are helpful, but that's all they are. (And don't get me going on how much money I think is wasted on training seminars, in the computer field and elsewhere.) Fri 03 Jan | soda | 'You can pick just about any topic and find someone giving a 1 to 5 day training seminar on it. Can you say someone who attends that seminar 'knows' the subject?' Unfortunately, in my experience, the person who attended the seminar 'knows' the subject much better than most people who claim they 'know' the subject on their resume... Fri 03 Jan | Tim Lara | Here's a variation on tapiwa's solution: Prob #1 SELECT t1.Sales_Id, t3.SalesRepName, t1.Color, t1.Make, t1.Model, t1.SalesDate FROM tblCars as t1, tblCars as t2, tblSalesMan as t3 WHERE t1.Sales_Id = t2.Sales_Id AND t1.Sales_Id = t3.ID AND t1.Color = 'Red' AND t2.Color = 'Blue' GROUP BY t1.Sales_Id, t3.SalesRepName, t1.Color, t1.Make, t1.Model, t1.SalesDate UNION SELECT t2.Sales_Id, t3.SalesRepName, t2.Color, t2.Make, t2.Model, t2.SalesDate FROM tblCars as t1, tblCars as t2, tblSalesMan as t3 WHERE t1.Sales_Id = t2.Sales_Id AND t1.Sales_Id = t3.ID AND t1.Color = 'Red' AND t2.Color = 'Blue' GROUP BY t2.Sales_Id, t3.SalesRepName, t2.Color, t2.Make, t2.Model, t2.SalesDate ORDER BY SalesRepName Prob #2 SELECT tblSalesMan.ID, tblSalesMan.SalesRepName, tblCars.Color, tblCars.Make, tblCars.Model, MAX(tblCars.SalesDate) AS LastSale FROM tblSalesMan INNER JOIN tblCars ON tblSalesMan.ID=tblCars.Sales_Id GROUP BY tblSalesMan.ID, tblSalesMan.SalesRepName, tblCars.Color, tblCars.Make, tblCars.Model HAVING MAX(tblCars.SalesDate) = (SELECT MAX(tblCars.SalesDate) AS LastSale FROM tblCars WHERE tblCars.Sales_Id = tblSalesMan.ID) ORDER BY Max(tblCars.SalesDate) DESC Fri 03 Jan |     | > Sorry, but saying 'SQL is easy you can learn it in a day' reminds me of the thread currently going where web programmers are complaining that people think they're just web page designers and their 12 year olds can give some tips. < I said 'Leanring basic SQL' is simple. Stored Procedures in PLSQL, ProC, ProCobol, TransactSQL, or any database API is difficult and I know that. Learning a database IMO is like learning an operating system. Sat 04 Jan | Curious | Problem 2: Surely it is last 'cars' not 'car' because the last two cars could have the same sale date? Sat 04 Jan | Albert D. Kallal | >>Problem 2: Surely it is last 'cars' not 'car' because the last two cars could have the same sale date? Excellent obseraton. In fact any SQL ace will also notice that I don't have a pk for the child table. That means that I cannot really use a top 1 conditon of 'date' to return only one record (which most sub-query conditions need). The basics of SQL are not too hard. However, because SQL is generally a non procedural language, thus getting data out of tables into the way your want can present many challenges. The fact that SQL is non procedural is both its strength, and its weakness. Many people coming from a clipper or older type database systems do find that quite a large mind set has to change to become effective with SQL. In fact, it is OFTEN easier to simply sit down and write code to solve a data problem (that is what most clipper folks always did anyway). I also find the same applies to people working with the Pick database system. Writing some code to find the last car sold can take LONGER via code as compared to SQL, but is actually EASIER from a brain cramp point of view! Programmers tend to think in procedures. Anyway, it was good see some people take a crack at those two simple problems. There are MANY solutions. For the 2nd problem (when last car was sold) the following works well. The reason why I prefer the following solution is that additional fields can be added, or removed from the query with ease (this means that I can include, or add additional columns from either table with out worry). I also don’t use any aggregate functions to return field values. This is important for future maintenance, and simple re-usability. I don’t want to have to use some max function on a set of fields, since all kinds of problems occur when trying to return lots of child fields (I purposely said return the color AND date etc because a simple max function will get the last date of sale. That is too easy, and in the real world that is not useful (what is useful is a nice table that can be sent to word to print mailing labels, or to excel). Thus I do believe that conditions in SQL that use aggregate functions like max is ok, but returning the values for use via aggregate functions is not useful at all! Thus, we really want something that returns a nice “table list” that can easily be sent to Excel, or to some report. After all, in the real world, we likely to be looking at a existing application, and almost for sure looking at a existing query. Chances are that the query and report already exist, but we want to change the criteria. So, I am REALLY looking for a solution that easily applies to EXISTING queries. Thus, like code…SQL is very much a “incremental” process. You work and massage the SQL towards your final goal. You work and massage code towards your final goal. That means that the solution should work on existing joins. In other words, you are likely to be starting out with a basic SQL datasheet that is a join already. I mean, you do copy, and paste and re-use existing queries right? Ok, I think point is clear hear. So, ok. With the above in mind, the first thing I want is a nice table listing of the sales man and the cars they sold. So lets write our basic every day join and ignore the “last car sold” problem for later (I have very very small brain, and thus really can only work with a small task at a time!!). : Lets write our join: SELECT SalesMan, Color, Make, Model, SalesDate FROM tblSalesMan INNER JOIN tblCarsSold ON tblSalesMan.ID = tblCarsSold.Sales_id; The above is real simple, and real basic. The result looks like: SalesMan Color Make Model SalesDate Albert Kallal Red Ford Mustang 01/03/03 Albert Kallal Red Ford Winstar 01/03/02 David Red GM Corvette 01/03/03 David Blue GM LaSaber 01/03/02 Bob Blue Dodge MiniVar 01/03/03 Bob Blue Ford Mustang 01/03/02 Now, that we have a our nice list to work with, lets restrict this list to only the LAST car sold. We do this by simply adding a where clause: That where clause is: where SalesDate = (select max(salesDate) from tblCarsSold where sales_id = tblSalesMan.id ) Gee, this whole things seems easier when we break it into steps..don’t it!! Thus, our whole SQL is now is the first part + 2nd part (this also means that the this solution works very well for IN-LINE SQL code. (because again we want to be able to ADD or modify the solution down the road..or later on in the code!!). Ok, the two together: SELECT SalesMan, Color, Make, Model, SalesDate FROM tblSalesMan INNER JOIN tblCarsSold ON tblSalesMan.ID = tblCarsSold.Sales_id where SalesDate = (select top 1 salesDate from tblCarsSold where sales_id = tblSalesMan.id order by SalesDate desc); The resulting table is: SalesMan Color Make Model SalesDate Albert Kallal Red Ford Mustang 01/03/03 David Red GM Corvette 01/03/03 Bob Blue Dodge MiniVar 01/03/03 The other nice thing about the above approach is that we can change the inner join to a left join. We will often want to list all salesman, and even those who have NOT sold a car would now appear in the list if we used a left join (and a is null for date). As also mentioned, the above solution allows us to apply the “where” clause to any given EXISTING SQL join we have. This is most likely to be most of the queries you work with. The above solution does of course assume that sub-queries are allowed. Most database engines out there do allow this. (JET and ms-sql for example do). MySql does not yet..but it is coming soon (gee, hope real darn soon!). As for the first solution. Boy, that one is wide open, and I get really weird answers all the time!! Anyway, again lets start with the original basic join statement. (hey, I can cut an paste the original!) If you ever have watched a real talented developer, you instantly notice how they cut and paste more code. These top notch developers do this, because they ALWAYS THINK in terms of re-usable solutions. The first few lines of code they type takes longer, but by the end of the day they have cut and pasted twice as much code as the guy sitting down the hall. Ok, so, I am going to whack the page/up key a few times and CUT AND PASTE my original SQL again. (I am saving time, or your are saving me money since I don’t really want to pay you again to write a query from scratch). Ok, here is query: SELECT SalesMan, Color, Make, Model, SalesDate FROM tblSalesMan INNER JOIN tblCarsSold ON tblSalesMan.ID = tblCarsSold.Sales_id; Ok, so now we need to add a where condition (remember, again..same issues as before, we want to be able to add, or remove conditions over time, and most important add to exiting queries. WHERE tblSalesMan.ID In (select sales_id from tblCarsSold where Color = 'Red' and sales_id = tblSalesMan.id) That gets us the red cars. Now add the blue: And (tblSalesMan.ID) In (select sales_id from tblCarsSold where Color = 'Blue' and sales_id = tblSalesMan.id) The above conditions are a bit long. You can also consider changing the “in” command to “=” and use “top 1”. Either way, most of the time the above select does get optimized by the query engine. However, while are a bit long, they can be added to our existing queries. Thus, if the boss now asks for red, blue and green car we can simply cut and paste the last condition, and change “Blue” to green. In other words, we can easily extend and add to the above conditions, BUT NOT modify the original query. Again, this is a obvious and important concept. May be we even want to throw in the last car condition into this solution here! In other words, I can combine my two examples into a 3rd with all the conditions. And I can do this whole 3rd condition via cut and paste (see I how I am now REDUCING time for additional solutions!!). Even more important is that we need a solution that works well via a GUI that is attached to code. I can take the above where condition and place it into a loop that concatenates that condition over and over several times grabbing a value from a multi-select listbox Take look at the following report screens: http://www.attcanada.net/~kallal.msn/ridesrpt/ridesrpt.html . You can see in the above that examples that I believe very strongly in supplying a GUI prompt screen for reports. I do not in any way believe that my end users should have to know, or even ever see SQL. It is a huge myth that end users need to build SQL. Users of that application have not asked for a new reports in about 2 years now. That means I have nailed the requirements right on. It also means that a few prompt screens for reports with the right options keeps a customer VERY happy! Other different solutions for the blue/red car given to me for this are: SELECT SalesMan, sum(Color = 'Red') as RedCount , sum(Color = 'Blue') as BlueCount FROM tblSalesMan INNER JOIN tblCarsSold ON tblSalesMan.ID = tblCarsSold.Sales_id group by salesman where RedCount < 0 and BlueCount < 0 Of course many times, the were clause can NOT used on fields that are a condition, so in the above the RedCount < 0 would have to be replaced with the 'whole' actual sum condition all over again. Of course the other problem with the above solution is that again aggregate functions are used, and you don’t wind up with a nice table listing that will no doubt get sent to a report writer or Excel. (in other words..it is of little use in a real application envornment). So, at end of the day, I actually agree that SQL is not really that hard. However, for a team of developers you do want to adopt some approach and standards as to how solutions are to be attacked. The end result is a lot more SQL and code that is extensible will be written. If the developers start writing more code..they also feel a lot better too. Albert D. Kallal Edmonton, Alberta Canada Kallal@msn.com Sat 04 Jan | Leonardo Herrera | The solution is to have an expert DBA in house, let the developers crank whatever SQL they want, but the DBA do the final reviews and approvals. I agree that anybody can do queries and stuff with SQL, but it takes a long road to become a DBA. Sun 05 Jan | Not sure if you were trying to impress me but... | No offense Albert, but your table definitions are terrible. If I ever had an interview question like that (and it was not specified that I could/was expected to add new tables or columns; ie that this is not the normal way you define tables internally) - I would assume that the company in question had little clue about db design and it would be unlikely I'd accept a position there. Also, we give out similar problems with a complete structure, and I can tell you that someone who solved problem #2 using a sub-select would be red-flagged, because so often a sub-select is going to be so much slower than (for example) an inner join. Sun 05 Jan | John McQuilling | Most of the DB vendors have limited time demos for NT or Windows 2000. They can be downloaded on a T1 or DSL speed link in a few hours at worst and some are available in the certification study guides (MS was). Down load the database install it and you have an environment to test in. Sun 05 Jan | tim | In case anyone cares, I ended up running MS SQL Server 2k on my laptop and a copy of IIS on my pc. I'm experimenting w/ asp and sql separately to get my feet wet and hope to hook the two together soon. Thanks for the recommendations/advice. Sun 05 Jan | Jan Derk | If you want to learn MYSQL and PHP you might also want to check out this walk through tutorial: http://www.webmasterbase.com/article.php?aid=228&pid=0 What Should I Do With My Life? | Thu 02 Jan | anonymous Good Article: http://www.fastcompany.com/online/66/mylife.html Thu 02 Jan | Tony E | Interesting Thu 02 Jan | Crimson | Very interesting article and I'm definitely interested in reading more about these people in the book. It takes a lot of guts to do what some of these guys did (i.e., walking away from jobs that were superficially successful to something that held deeper meaning for them). Not sure if I could do the same. Thu 02 Jan | anonymous | Have a search on Erich Fromm for some refreshing toughts. Thu 02 Jan | programmer | Honestly, the book sounds like more over-caffeinated B.S. from the self-appointed cheerleader of the dot com boom. Now that the dot coms have crashed, what's Po Bronson going to write about? People who walked away from high-powered careers. What amuses me about the excerpt from his book is that he talks about people who walked away from high-powered careers, as if he's trying to show that they learned 'what really matters' -- but the way he describes their situations shows he's still mightily impressed by the academic credentials, high salaries, and high-octane jobs his subjects had. There's an interesting paradox at the heart of books like this -- they claim to be about people who realized that competitiveness and money and success and fast-paced lifestyles don't matter, but the subtext of the things these people are being celebrated for is that they 'figured out the secret to happiness,' or they are 'more successful' because they abandoned conventional notions of success.' You get the feeling that people like these just found another arena to be competitive in, when they realized they couldn't win in their more 'conventional' jobs. So despite the claims that these people have given up the race for 'success,' if you read these books, you still feel that you are having somebody else's success shoved in your face. This book purports to be about the personal transformations of the people he writes about -- but to me, it appears that it's more about the 'Personal Transformation of Po Bronson Now That I Don't Have The Dot-Com Boom to Rave About.' Thu 02 Jan | Robert Moir | Why is someone who had a high powered high stress job and who walked away from it to do something that really matters man, like teach and give back to society man, more of a hero than someone who has always taught? (Of course assuming you agree that being a teacher is all that in the first place) wouldn't the guy who completely avoided the high stress high power job in the first place and went straight to being a teacher actually be more successful? They managed to see the light without making a mistake first, after all. Thu 02 Jan | Sebastian | IMHO, this article demonstrates the ever growing concern of our changing society (in North America atleast) about our dissatisfactions in what we do with our lives. More and more people are searching for some kind of guidance and epiphonies to feeling in a place where they belong. (my personal insert) At age 28, in Canada, and doing software development for the past five years, it recently hit me to wonder 'what I have accomplished with my time on this planet. What do I have to show for?' And so, I've started my own software company, doing what interests me. In the hopes that this venture will aid me to focus more on becoming more satisfied in doing more what I want end enjoy with 'my' time, and with helping others using my tools and services. I guess its some sort of 'share knowledge and give back to society, yet still make some profit so you can feed yourself'. And so, now, while I sit here, unemployed (due to layoff), and work on attributes of my own life and work, it brings a feeling of ambition to have a chance at a life you can be proud of. I praise 2003 to be a year of continued appreciation and new success. The truth, and the way it is: http://www.monbiot.com/careers.cfm Thu 02 Jan | Devil's Advocate | Robert - cf 'Prodigal Son' Thu 02 Jan | | This is my favorite quote from Po Bronson's article: 'Carl Kurlander wrote the movie St. Elmo's Fire when he was 24. For years afterward, he lived in Beverly Hills. He wanted to move back to Pittsburgh, where he grew up, to write books, but he was always stopped by the doubt, Would it really make any difference to write from Pittsburgh instead of from Beverly Hills? His books went unwritten. Last year, when a looming Hollywood writers' strike coincided with a job opening in the creative-writing department at Pitt, he finally summoned the courage to move. He says that being in academia is like 'bathing in altruism.' Under its influence, he wrote his first book, a biography of the comic Louie Anderson.' Since when was writing a celebrity bio 'bathing in altruism'? Thu 02 Jan | Bella | > Why is someone who had a high powered high stress job and who walked away from it to do something that really matters man, like teach and give back to society man, more of a hero than someone who has always taught? B/c the person who walked away made a conscious choice to be poor and give back. Someone who always has taught may be perceived as doing that b/c he had no other options. ie: Not the same caliber. Very few people conscious chooose a poverty level career when they have other choices. Fri 03 Jan | Robert Moir | Valid point Bella and one I was aware of. OTOH a lot of these books and features - for this isn't the first time I've seen writing on this subject - deal with Dot Com businessmen and women. Hardly paragons of business sense. Perhaps it was they who didn't have a choice about what to do next. Fri 03 Jan | programmer | Bella, From my experience (people I've known who are teachers) people do not go into teaching because they have no other options. They DO have other options, and they DO choose to teach. I suspect that many people who pooh-pooh teaching and think it's a last-resort career for losers, would be surprised at how difficult teaching is. And would be surprised at how many people go into teaching for genuinely altruistic reasons. Maybe you don't actually the things you said -- perhaps you were just describing a 'perception' -- but in any case, the perception is dead wrong. I'm not a teacher, by the way. Fri 03 Jan | programmer | In the second to last paragraph of my above post, I meant to say, 'Maybe you don't actually believe the things you said ...' Fri 03 Jan | zaphod | I picked up this book yesterday, and have read through three quarters of it. Like many out there, this is a question I have struggled with for quite some time. Contrary to what others have suggested from the FC article, this book does not promote the choices of people who have walked away from high powered careers to do something meaningful. While there are stories like that, the amount of different stories told in the book is astounding. This book is so much more and yet so much less than I had anticipated. So much more due to the sheer variety of stories that are told, and the fact that they are all true stories - the impact that some of them had on me surprised me a great deal. Yet, the book is so much less than I had hoped for exactly the same reason. These are real stories, and real life is complicated. Many of the stories in the book end the question still hanging, choices left unmade, and great uncertainty in the minds of those who have made their choices. There are no simple answers to real life problems, and Po Bronson does not attempt to push meaning, or morals, or guidelines out of these stories. I had hoped that the book would reaffirm the decision that I need to make. If anything, the opposite is true and I am more confused than ever. Yet, while the question posed in the title is still far from answered, I am left with lots to think about, and at the very least the book has provided hope through the knowledge that I'm not alone. Fri 03 Jan | Herbert Sitz | Zaphod -- I know what you mean. My undergraduate major was Philosophy. I think I chose that because I thought that there was somebody else that had some wisdom they could use to tell me what it was I was supposed to do. Nope, they can't, things don't work that way. I was at least glad to learn that at a fairly young age. Fri 03 Jan | 11010 | Bella didn't say that people who went directly into teaching didn't have any other options, he said they would be perceived as having no other options, whereas the CEO who becomes a teacher obviously had other options. >> I suspect that many people who pooh-pooh teaching and >>>think it's a last-resort career for losers, would be surprised at >>>how difficult teaching is. I disagree. Teaching is NOT difficult. Certainly being a good teacher is difficult, but so is being good at anything. For the most part, teachers (in the USA) just follow recipes learned in education school. Teaching is a last-resort career for losers. People become high school teachers because they want summers off. People become college _teachers_ because they weren't smart enough to have a research career that went anywhere. People become elementary school teachers because they are freaks. People become technical trainers because it is easier than actually doing technical work. Fri 03 Jan | Brent P. Newhall | zaphod: And I think that's good. Real life doesn't have simple answers (usually). A book about difficult life choices that manages to convey that fact has done an impressive service to its subject. Major life choices are *hard*. But worth it. Fri 03 Jan | programmer | 11010 -- Just curious -- what is the source of your hostility toward teachers? I'm trying to imagine what sort of person you are, hiding behind your anonymity and spouting such venom at an entire profession of people who do great work for an ungrateful society. The images of you that I'm coming up with are not pleasant. A profession is a core component of someone's identity -- by demeaning it in such a way, you attack them in a very hurtful, personal way. I suppose that's funny to someone with the kind of heart you have. Anyway, with your point of view, I think programming could easily be demeaned in the same way you have demeaned the teaching profession. You could describe the people who drift into programming in a way very much like you described the poor losers that you think fill the teaching profession. People drift into this profession for various reasons -- not always because they have a passion for it. And most programmers aren't geniuses. But since you're a programmer, apparently, you direct your bitterness and rage only at other professions. Fri 03 Jan | 11010 | >>I'm trying to imagine what sort of person you are, hiding >>>behind your anonymity and spouting such venom at an >>entire profession of people who do great work for an ungrateful >>society. programmer, are you serious? Where did you go to school? I went to public elementary and high school in the midwestern USA. Most teachers are not passionate, capable saints like Jaime Escalante in 'Stand and Deliver.' Most teachers truly did become teachers because there are few other occupations (besides maybe, uh, computer programming) with such a low barrier to entry. Teachers on average do NOT do great work. 90% of my male teachers became teachers because they wanted to coach high school football or hockey. The females became teachers for who knows what reason. You need a C- average and a teaching certificate to become a teacher. You don't need any domain experience. If you had a good experience with your secondary education, good for you - you are one of the lucky ones. also, I maybe posting anonymously, but I left an email address... 'programmer.' ... Sat 04 Jan | zaphod | > >Major life choices are *hard*. But worth it. > So true. But that doesn't stop me from looking for the easy answers... :) Sat 04 Jan | Alex Chernavsky | I could never be a teacher today. ========================== Rude Chicago Students Chicago Sun-Times — February 11, 2001 BY ROSALIND ROSSI, EDUCATION REPORTER One new Chicago high school teacher, a native of the Middle East, was astonished when she discovered that her students had put blue chewing gum on her chair and stuck funny signs on her back. Another teacher, born in Hungary, couldn't believe it when she called a high school student to the blackboard and he insisted that his 'tutor' – another student – accompany him. The teachers are part of the Chicago public schools' latest answer to an escalating shortage of teachers. The Chicago schools posted recruitment notices on the Internet, and 1,300 men and women from 21 countries responded. 'We had no idea what we have to face,' said Liza Koves, a teacher from Hungary. The teachers say they are stunned by the American students and their talking, tardiness, absenteeism, laziness, vanity, foul language, apathy and general disrespect for them. ============================= http://www.astrocyte-design.com/interests/chicago.html Sun 05 Jan |   | If I ever have kids, they are going to a school where nuns carry rulers.