Sunday, February 29, 2004

Re: communicating to a modem

There was this thing I'd tried back in junior college days. Basically, you connect to a friend's computer using hyper terminal. On the server side, you punch in the command ats0=1 or something like that; don't remember exactly. Then a friend would dial into my computer using the modem and we could transfer files. It was a good way to transfer files too big to mail. And it was pretty quick too (about 4 KB/s... yeah that's quick... quicker than dialup downloads :-))

I've a question... is there a program which I can use to make / answer telephone calls? Something which runs in the background and tells me when the phone rings (assuming I don't have a regular phone connected) so I can answer it using the computer itself... listen on the sound blaster and speak into the mike connected to the sound card. I tried the phone dialer one time but I think it just dialed... so wasn't a complete telephone call. Are there any settings which have to be tweaked or something like that?

communicating to a modem

when i first did tried this it definitely was a "eureka" moment for me. i doubt how many of you broadband guys (yup .. i am still on dial-up) still have a modem though.

basically i'll show you how to issue commands to your modem. the modem can be programmed to perform various tasks.

goto start>>programs>>accessories>>communications>>hyper terminal. you will be asked to create a new connection. just give any name you want. in the connect to option give COM1 or the similar COMx port where your modem is connected. in the next window, you can set Bits per Second to 9600. after ok you are connected to your modem.

now you have a blank text window staring at you and thats all. type in at and voila, you get a reply OK. that was your modem talking. in modems, standard commands have been defined. this protocol is called AT-command set. the command you issued earlier was attention (at). you asked the modem for it's attention. just a sort of are_you_present check. another command is ATDT=xxx which will dial through your modem to the number xxx. there are many other codes which you can google for.

also the modem can reply in numeric or verbose format to your commands. numeric commands are well, numeric like 0 1 2 3 .. each have meaning as described per command. verbose commands are like OK CONNECT RING BUSY etc.

so you can create an app, which communicates with the COM port, and listens for the above strings, after passing the commands. and can talk on the phone through your the pc. there are two modes of communication with a modem actually, a data mode and a command mode. in the command mode the modem respods to stuff like AT etc. when you convert to data mode, everything you send to the modem is just passed through. ATA, ATH, +++ are some more useful commands. i haven't explored or tried this much so not very sure of some things.

and it gets better. for mobile communication there exists a AT+ command set. instead of a normal modem you can connect a Nokia 30 GSM terminal to the pc. now just insert a sim card and communicate wirelessly. the AT+ command set is a super set of the AT commands. advanced commands exist for operations supported by mobile devices, like sending SMS'es. also can use advanced, faster mobile protocols like GPRS etc.

i have just given a brief overview of the stuff within limitations of my knowledge. please google further for more info. also i suggest that you read more on the Nokia 30 terminal. also it seems that Nokia 31 terminal has been released. dinesh could also add on to this blog.

in linux there is an app minicom which is the equivalent of hyper terminal.

thinking in flash

How well do you guys know Flash? Everything on there is done in Flash. I'm just wondering how much "programability" Flash offers. Cause it looks quite compilicated to me.

i used to do quite some stuff in flash long back. quite some programming also. one very good site for flash resources is flashkit.com. did a few projects which gave me a lot of satisfaction. there was a time when i was much better at flash than at java actually!!.

i guess i was comfortable with flash because i thought of it in a more OOP way. flash == OOPS with images. i'll explain further.

there are a few object types. movie and button are the most important. when you create a movie or button symbol you are in effect extending the basic classes to create your own. the symbols you put on stage are also just called instances. you also have to give instance names to whatever instances you put on stage. the one in the library is the extended class.

there are some super methods/variables you have when you extend the movie or button classes. these are the api calls you can make to manipulate your instances. stuff like x._alpha etc. the entire api is very well documented in flash help.

another very important part in flash is handling events. buttons and movie instances also have associated events, which you can trap and use. stuff like
on (press){
getUrl("products/index_warli.html","_self");
}
for buttons. basically event handlers/delegates whatever.

in programs you start from a main and flow through a program and create instances, call methods etc. in flash you flow through the frames. simply stop(); at a particular frame, and jump with gotoAndPlay(5); on some event etc to move through the animation.

i would say think partly OOP!!

Saturday, February 28, 2004

Good Assertions example

I was just looking over some code I had written sometime back and came across something that I thought would be a good area to use Assertions.

This has to do with using enums. Enums in c/c++/c# (and coming soon to Java in 1.5) are types which consist of some named constants. They are much preferable to just constants or "magic" numbers cause they are much clearer and are categorized.

Anyway, here's a simple ex of using it...

enum Color
{
    Red,
    Blue,
    Green
}

If your method takes a Color argument it can work with the values set in this enum.

private void ChangeBackground( Color color )
{
    switch ( color )
    {
        case Color.Red:
            // Change background to red
            break;

        case Color.Blue:
            // Change background to blue
            break;

        case Color.Green:
            // Change background to green
            break;

        default:
        // Don't do anything
        break;
    }
}

This is fine, but what if sometime in the future someone adds a new value to Color? That switch statement will drop down to the default case. And you won't even notice it.

So it would be good practice to add an assertion in the default case.

default:
    Debug.Assert( false, "ChangeBackground(): Color " + color + " was not recognized" );

This is basically protecting yourself from your own code. In this example, it's not such a huge deal if it falls through to the default case... just that your newly added color won't be handled. But in other cases it might be more critical.

Re: levitated.net

Pretty cool stuff.

How well do you guys know Flash? Everything on there is done in Flash. I'm just wondering how much "programability" Flash offers. Cause it looks quite compilicated to me.

Friday, February 27, 2004

levitated.net

you guys have to check this out

http://levitated.net/
http://www.levitated.net/daily/index.html

the stuff here really blew my mind!! simply awesome

dinesh.

Wednesday, February 25, 2004

We Are Morons: a quick look at the Win2k source

http://ww.kuro5hin.org/story/2004/2/15/71552/7795

No doubt you've heard about the Win 2K/NT source code leak. This article just takes a superficial look at it.

I'm guaranteeing you'll shit your pants laughing at some of the source code comments.

Re: Return of the Assertions

assert can be enabled/disabled during running code however, with the -ea , -da options.

how is it in .net ?


The .NET languages don't have an assert keyword. The framework has a class called Debug in the System.Diagnostics namespace which has an Assert() method.

The CLR depends on the presence of a "DEBUG" symbol to JIT compile Assert calls. You use the #define statement to define the symbol -> "#define DEBUG". The methods in class Debug have an attribute called Conditional associated with them. Ex...

public class Debug
{
[Conditional("DEBUG")]
public static void Assert( bool condition );
}

At compile time (i.e. JIT compiling to native code from IL code), when it comes across the Assert() methods, it sees that it has the Conditional attribute associated with it. It checks to see if the DEBUG symbol is defined. If yeah, it goes ahead and compiles, else skips it. So, no increase in code size and doesn't hurt performance.

Actually .NET has another class called Trace. That class also has Assert() methods, which essentially do the same thing as Debug Asserts. The difference is that Trace uses a different symbol. It uses "#define TRACE". Also, the Conditional attribute on those methods are passed "TRACE".

They have two cause you want some assertions only during debug time, (Debug.Assert()) and others ALL the time, even on release builds (Trace.Assert()).

another question (probably the dumbest however) i had is whether FooBar is supposed ot mean anything. as i i have seen so many FooBar examples, i wonder if it was some historic programming whatever. u guys know what its supposed to signify or do i just need a break !!

I don't really think it means anything specific. It's just something that's been passed from gen to gen in program examples and it's stuck. Here's a def from the "The Jargon Dictionary" http://info.astrian.net/jargon/terms/f/foobar.html

There is another acronym that sounds the same - FUBAR (Fucked Up Beyond All Recognition)

Friday, February 20, 2004

Binary finger counting

http://www.johnselvia.com/binary/bin1-7.html

The next time you're pissed off at some one, send em a 4... unless your in England. In that case go for a 6.

Re: iptables

there is actually so much stuff i can ask you on linux actually. did u try kernel 2.6? i tried compiling but on restart get major errors.

Nope... I guess I didn't have the motivation to do it. But I intend to do it sometime soon... maybe after my mid-sems get over. :-)

but i was still able to ping myself !! any ideas.

I tried it and it worked for me... it dropped the ping packets. Don't know how you could still ping yourself! Do you have the right entry corresponding to 127.0.0.1 in your /etc/hosts file? Or if you were connected to the internet and pinged your own machine, the packets could've originated from the dynamic ip (assuming dialup) and would've passed... not sure about this though... don't know!

actually yes and no. firstly there are two parts, packet filtering and actual rules which say what action to be performed to specific packets. packet filtering capability has to be enabled in the kernel. networking protocols etc are all specified in kernel options during compilation. during normal operation there is the iptable app which sits on top and performs various firewall operations based on the packets which the kernel gets. hrishi, correct me if i goofed in the above explanation.

You're pretty much right. In plain English, packet filtering is built into the kernel. So one could say that it is very much a part of linux. iptables is just a tool which tells the kernel what to accept and what not to... In other words, the decisions about packet filtering are made by the kernel (using the rules in the chains) and not by the program iptables. Iptables isn't a daemon...

your iit festival might have been a blast. did you participate/organise?

Nope... just enjoyed! I'm pretty lazy about these things! :-)
There was this lecture by Prof. Kevin Warwick of University of Reading, UK. Don't know if you guys have heard about him; he'd implanted a (actually 3 if I'm not wrong) chips in his hand... and became the first cyborg. It was pretty good. I'll write more about it some day.

Re: iptables

Long time no see... :-)

great to hear from you hrishi. and even better on getting some good linux tips. try and post a bit more often. maybe just links of some good resources/features on linux or anything for that matter.

there is actually so much stuff i can ask you on linux actually. did u try kernel 2.6? i tried compiling but on restart get major errors.

your iit festival might have been a blast. did you participate/organise?

Iptables is an excellent packet filtering tool.

i read up on the link you sent and tried one example from the iptables-HOWTO. the example was to disable ping by
# iptables -A INPUT -s 127.0.0.1 -p icmp -j DROP
but i was still able to ping myself !! any ideas.


Isn't a firewall a separate piece of software? Separate from Linux?

actually yes and no. firstly there are two parts, packet filtering and actual rules which say what action to be performed to specific packets. packet filtering capability has to be enabled in the kernel. networking protocols etc are all specified in kernel options during compilation. during normal operation there is the iptable app which sits on top and performs various firewall operations based on the packets which the kernel gets. hrishi, correct me if i goofed in the above explanation.

Thursday, February 19, 2004

Return of the Assertions

i was chatting with mohn , and we ended up discussing assertions again. the discussion ended with mohn suggesting disabling assertion at compile time.

i looked it up, and realised that it is not possible to disable assertions during compilation. (in java atleast). an example in javaworld, explains the point.

assertions were included in java 1.4. so all assert containing code is compiled with the flag -source 1.4 . suppose you do not include this option and try to compile code which contains the assert keyword,
Foo.java:5: warning: as of release 1.4, assert is a keyword, and may not be used as an identifier
is the error you'll get.

assert can be enabled/disabled during running code however, with the -ea , -da options.

how is it in .net ?

another question (probably the dumbest however) i had is whether FooBar is supposed ot mean anything. as i i have seen so many FooBar examples, i wonder if it was some historic programming whatever. u guys know what its supposed to signify or do i just need a break !!

Re: Code Review

please confirm what i understood. as many classes in one package get converted to one exe/dll file. also the size of these exe's/dll's must be very small comparatively. say a few 5-10kb for small apps plus some extra 10+ kb for the metadata.

Yes... many classes can be compiled down to one exe/dll. The key word is "can". You can compile each class to it's own dll if you want to and then reference those dll's in an exe.

I'm not sure about the size of the exes/dlls. For my CDPlayer app, which had 9 source code files, several images and a resource file, the exe came out to 144 kb. I'm not sure how the assembly stores the metadata exactly, but you can view it using the il disassembling tool (ildasm). But the metadata shouldn't take up too much space... I wouldn't think.

mohn told me that code on one (earlier) .net version could not be run on the newer version because of configuration file stuff. could you (mohn) explain more on this. also in .net, are certain parts of the api removed for next versions, or carried about like in java.

When you create an application with one version of the framework, your app will only run if that ver of the framework is installed. That info is contained in the exe/dll metadata. So again, taking my CDPlayer app as an ex. I created it using v 1.0. You (Rahul) have v 1.1 on your machine. So if you download my .exe and try to run it, you will get a message saying that the correct ver of the framework is not available.

Now, this is NOT to say you cannot run it on other framework versions. All I have to do is create a config file and give that to you along with the exe. When you run the exe, the CLR will look at the config file (this is done automatically), which will say that my app is compatible with v 1.1 and it will execute.

So this gives you the flexibility to limit your apps to certain versions if you want to. Only two versions of the framework have been released so far v1 and v1.1. And as far as I know there aren't any breaking changes. But in the future, if something does change, you can ensure that your app requires a certain ver of the framework to execute through config files.

Re: Code Review

In .NET, everything comes down to an assembly. That assembly - be it .exe or .dll - can contain any number of classes. You deploy and execute that. You can just copy a .exe from one machine to another and execute it. As long as the machines have the CLR, nothing else is required. No registering anything, dll hell etc...

please confirm what i understood. as many classes in one package get converted to one exe/dll file. also the size of these exe's/dll's must be very small comparatively. say a few 5-10kb for small apps plus some extra 10+ kb for the metadata.


does every new release of Java update the JVM?

i am answering with what i have experienced. in java, each 1.x version is also very major. and generally takes about a year or so for the next version. so by that time there are usually major updates in jvm performance as well. the 1.4.x (suppose) are minor updates. mostly bug fixes.

also all java code is always backward compatible. even though a particular method might be deprecated, you can still use it though it will raise an warning. also all java legacy code will work even on the latest versions. this is actually a slight point of difference between developes. because you can still use deprecated code, some do not agree with this functionality. this is more of a financial decision by sun.

mohn told me that code on one (earlier) .net version could not be run on the newer version because of configuration file stuff. could you (mohn) explain more on this. also in .net, are certain parts of the api removed for next versions, or carried about like in java.

Wednesday, February 18, 2004

Re: iptables

Anyways... another example of the high customizability of linux. I wanted to set up a custom firewall rule on my computer that would block all requests to port 80 of any server that is outside the campus. Just a one command job. Iptables is an excellent packet filtering tool with several updates over it's old cousin (or should I say ancestor) ipchains

Isn't a firewall a separate piece of software? Separate from Linux?

BTW, another spoof on google.com. Check out googirl.com.ar :-)

They seem to have made booble much more interactive.

Re: Code Review

also for gui based apps it is very odd for a cmd/shell window running in the background in which the java command was given. so there also exists a javaw command which will start the gui app without the cmd window

Java was designed with the intention of being run on set top boxes right? Then it moved to the server and applets. I suppose because of this it's not that "client friendly". How often dyou see java apps on the desktop anyway?

these problems are further magnified as in java each class is in a seperate file. (which i think is not the case for .net)

Yup, it's not the case in .NET. In java your packaging structure is interelated with your file/folder structure. That's not such a great thing. You don't have freedom to organize it the way you want. I wonder why they chose to do that.

metadata has been introduced in java 1.5. i do not know if metadata will change this scenario. mohn could tell us more on .net packaging, and probably a lot more on meta-data.

From a little I've read, the meta-data functionality in java 1.5 is basically attributes. So you can add attributes and the jvm will inject code when you compile it.

I wrote some stuff about meta-data in my prev gigantic posts on .NET. Basically, it is some info that will be used by the runtime (clr/jvm). For ex, in .NET, you can include an attribute for Serialization for a class. When you compile your code to intermediate code, the compile will also metadata saying that this class has been marked (flagged) for serialization. The runtime will inspect this metadata at run-time and automatically serialize your class when you want it to. This is something the runtime itself implements. You can also include your own custom attributes and then use reflection to inspect it.

In .NET, everything comes down to an assembly. That assembly - be it .exe or .dll - can contain any number of classes. You deploy and execute that. You can just copy a .exe from one machine to another and execute it. As long as the machines have the CLR, nothing else is required. No registering anything, dll hell etc...

Probably a dumb question... but does every new release of Java update the JVM? Or is it just the class library that gets updated? For java 1.5, I'm pretty sure with attribute functionality, generics etc..., the jvm will have to be updated, but what about for others - like minor updates?

iptables

Long time no see... :-)

Anyways... another example of the high customizability of linux. I wanted to set up a custom firewall rule on my computer that would block all requests to port 80 of any server that is outside the campus. Just a one command job. Iptables is an excellent packet filtering tool with several updates over it's old cousin (or should I say ancestor) ipchains.

Here's a link which would give a lot of info about iptables.
http://www.linuxguruz.com/iptables/howto/

BTW, another spoof on google.com. Check out googirl.com.ar :-)

Re: Code Review

as mohn pointed out, in java jar files are used to package applications. actually jar files are just glorified zip files with a slightly different compression.

in java i can execute a jar file directly. by the command
$ java -jar MyJarApp.jar

inside the jar packages there exists a manifest file. the manifest file declares the entry-point or main method containing class. this class is executed first.

there are no exe's or dll's in java. for java apps, generally batch or/and shell files are used so that the client does not have to actually java. i have noticed many apps which provide both batch(.bat for win) and shell scripts(.sh for *nix). sometimes exe files are created which spawn java processes. but this is platform dependent.

also for gui based apps it is very odd for a cmd/shell window running in the background in which the java command was given. so there also exists a javaw command which will start the gui app without the cmd window. to try jar and javaw try the following
goto j2sdk install dir
there goto demo>>jfc>>Stylepad
$ javaw -jar Stylepad.jar

i feel that this(jar packages) is a problem for desktop apps. but for server apps its fine. these problems are further magnified as in java each class is in a seperate file. (which i think is not the case for .net). in libapp i have extensively used inner classes for all gui widget handlers. so i think the class file count reaches about 100, which is a lot.

metadata has been introduced in java 1.5. i do not know if metadata will change this scenario. mohn could tell us more on .net packaging, and probably a lot more on meta-data.

by the way, jar == java-archive. for j2ee, there also exist war and ear files. war == web-archive. a war file can be deployed within any j2ee compliant web server, and will run. war files mostly contain jsp's, servlets etc. ear == enterprise-archive. ear files can be deployed within application servers. ear files mostly contain ejb's.

Tuesday, February 17, 2004

Re: Code Review

Downloaded, but haven't followed your instructions yet... I will (soon).

Just wondering how exactly are Java apps deployed? When you compile your .java source files, a .class file is generated. In a large project with 100s of files (your app actually has quite a lot of files) how would you package that? You don't have .exe's and .dll's in the java world do you? I have heard of .jar files, but it seems more like zip than an exe.

Code Review

i am posting some code i had written last year. libapp was a library application written in java (obviously). the app source has been put up on www.avivaint.com/temp/libapp.zip. this is a temporary locaton and i'll keep it for a week.

the app consits of a GUI in swing which is a client app. a rmi server is also started to which we connect. hence the app is distributed. also on the server-side jdbc calls are made to the database in which the actual data is persisted. libapp requires some setting-up and compiling. these instructions you'll find in the README file included in the zip file. as lazy as i am i have not tested the app recently. so let me know of any errors in setting-up/compiling or bugs.

i would really love suggestions for improvement on the code so lets keep blogspot a little busy.

and yes.. all the code is released under GPL so feel free to use/distribute.. (i.e if you find anything useful to do with it)

Sunday, February 15, 2004

User Friendly is tops

http://ars.userfriendly.org/cartoons/?id=20040208

http://ars.userfriendly.org/cartoons/?id=20040215

Re: Assertions!!

static final boolean assertionsEnabled = ;

if( assertionsEnabled )
assert expression1;

during debug build, the static final boolean is set to true. so that all the assertions can be enabled.

when compiling for release build, set the boolean to false. now since the boolean is final staic, all the assertion code will be unreachable. hence javac will result in no bytecode being formed for the assertion code. this will have a small perf benefit compared to when assertion bytecode exits and assertions are disabled.

RE: movie recommendation

ROTR was released last week or so..
what was most important was that they were able to maintain the movie's "feel" throughout the trilogy, unlike matrix.

Saturday, February 14, 2004

RE: movie recommendation

definitely not lord of the rings but a good movie none the less...

Has ROTK come out in India yet?

Re: Assertions!!

is it not necessary to be checking whether the values of data objects will be in a consistent state even during release code? or in that case will you change these assertions in the model to exceptions in the controller or something like that?

I see your point. You should be testing that those assertions are true everytime the code is run... even in release mode. Maybe it would be a good idea to leave some assertions - some extremely important ones - enabled and disable others. And if and when an assertion does fail, to catch the AssertionFailed exception gracefully and display a message to the user.

But my feeling is still that assertions are a debug tool. They are there to help the programmer from his own errors. Any input errors made by the user should be handled by exceptions. Same with invalid arguments passed into methods by other programmers... exceptions are the way to go.

You will test your app in the debug build with all assertions turned on. You run a whole bunch of tests and no assertions fail. You are reasonably satisfied with it... to some degree. That allows you to turn off the assertions.

In the code you provided, the static booleen variable is there to turn off assertions. Why would that be there, if it wasn't optional? As in, with the intention of being turned off at some point. Some books say to "assert everything". You can take that to mean whatever you want... but it is essentially saying use it liberally. If you do that, it will slow down your app. So to speed up performance, you would need all them assertions disabled when you release it.

Are you Eligible?

From: careers@wipro.com
Subject: Wipro University Program
Date: Sat, 14 Feb 2004 18:47:11 +0530 (IST)

Wipro University Program
Calling BE / BTech / MCA Class of 2003 only.

Wipro University is an off-campus program for fresh graduates. On joining, the trainee will go through an intensive training which will prepare him/her to take on the ever emerging challenges in IT. The program combines conventional classroom training with live assignments, and the trainees are paid a stipend during the training period.

Eligibility Criteria:
1. Graduates from class of 2003 only
a) BE / BTech (Computer Science/ Electronics / Telecommunications / EEE / Information Technology / Instrumentation / Information Science)
b) MCA

2. An aggregate of 70%. (Aggregate calculation is: - Total marks obtained in all semesters divided by Maximum marks). Please note: - An aggregate of 69.9 will not be considered as equal to 70%.

Selection Process: The exercise starts with a written test (Objective type) covering Verbal, Analytical and Technical sections. The technical section will cover questions on computer fundamentals, basics of fundamental languages like C, C++ and Java.

On clearing the test, candidates will attend a Technical and an HR interview.
The selected candidates will undergo a 3 month training (classroom and On the Job) on specific software domains. After successful completion of the training period, they will be placed on projects.
The selection process will be held in the month of March, 2004. The candidates short listed for the written test will receive intimation though E-Mail.

HOW TO APPLY?
Candidates, who fulfill the above eligibility criteria, can apply Online under
the Off-Campus section at http://careers.wipro.com, before February 16th 2004.

Please forward this mail to your eligible friends.

STRATEGIC RESOURCING TEAM
WIPRO TECHNOLOGIES

movie recommendation

i do not watch many movies, let alone hindi ones. but i would recommend "khaki". i saw it on cable. its a fight against the corrupt system type movie with some nice twists. definitely not lord of the rings but a good movie none the less...

Friday, February 13, 2004

Re: Assertions!!

i understood all you said warlier on debug/release builds. i'll rephrase my earlier question.

is it not necessary to be checking whether the values of data objects will be in a consistent state even during release code? or in that case will you change these assertions in the model to exceptions in the controller or something like that?

this was one example i read in this article , the prequel to the earlier article i posted on assertions. it explains more on the implementation of assert in java.
the part i wanted to draw attention to is how to remove assertion code from your class (maybe for not so professional release builds!!)

" The assertion facility does not provide a direct means of conditionally preventing the inclusion of assertion code in a class file. The Java Language Specification, however, provides a standard idiom for achieving conditional compilation in Java. Applied to assertions, the idiom looks something like this:

static final boolean assertionsEnabled = ;

if( assertionsEnabled )
assert expression1;

Since the variable assertionsEnabled is static final, when the value is false, a compiler can detect that the assert statement is unreachable and can remove the assert statement code. "

Re: Assertions!!

i felt that your class invariant code was very dependant on assertions. if assertions were disabled then all the checking for data state consistency would be lost. is that not required during the normal working of the code and not only during debugging?

Right, assertions will be disabled when NOT debugging. When I say debugging... I really mean development, i.e. before release. You will generally have two builds of your code - Debug and Release. The Debug build has all the assertions enabled. In Release, it will be disabled. Assertions are expensive (depending on how much you use them)... so you will want to disable them before releasing for performance reasons as well.

As I said last time... assertions are for YOU. They are there to protect you from yourself. You are writing code in one method that somehow changes the class data, for example age, to 70, when the limit is 65. If you didn't have the assertions there, you wouldn't notice it. With the class invariant called in each method, the assertion will fail and you will see it.

Class invariants are akin to Unit Tests in a way... but integrated with the class. As you keep adding new code, you want to make sure that everything you wrote before works. You want to be sure that your new code didn't break anything before. Whereas Unit Tests test from the "outside" - as a consumer to the object, class invariants test from the "inside" to make sure the data is always in a valid state.

What happens when an assertion occurs? You get some error message telling you what line the assertion failed on and some other data. You should be the only one seeing that. Not the user. The user should see a friendly message.

What are your thoughts on assertions vs exceptions? When/Where dyou feel they should be used?

Thursday, February 12, 2004

Re: Assertions!!

thanks for the code on class invariant..

but that got another question into my head. (i guess i just can't stop asking dumb questions)..

i felt that your class invariant code was very dependant on assertions. if assertions were disabled then all the checking for data state consistency would be lost. is that not required during the normal working of the code and not only during debugging?
if yes then your earlier statement of assertions only for debugging is not so assertive!

Re: Assertions!!

I briefly skimmed through the article. He makes some good points, but I disagree with him regarding where to use assertions.

are you trying to imply that assertions should be used only for debugging?

Yes...

Personally, here is how I view things. There are three types of "customers" of your code (depending on what you are developing). First, is the end user of your product. Second, is a third party developer who will use (program against) your code. And finally, third is YOU or your team.

If you are developing a library or component, then most probably you will NOT have an end user. A third party developer will be your customer. He will be using your library in his app. And conversely if you are developing a full application, you will not have a third party guy programming against it - unless it is something like MS Word which has that capability.

Anyway, I feel that assertions are to be used only for YOU and your team. Neither the end user nor the third party programmer should face assertion faliures. They should instead face exceptions. For the end user, you should display a nice user friendly message and I guess log it. For the third party developer, just throw the exception (which should be in the documentation). Assertions should only be used during development. Before releasing your code, you should remove them (not literally - there are compiler options where you can tell it to just ignore them).

Assertions are there to protect you from yourself. When you sprinkle assertions all over your code and you run it, if you have made any errors, you will notice it when the assertion fails. So, you do you testing and everything runs fine. You release the product to the customer. One day your app shows error messages or stops responding or whatever. He calls you up and tells you he entered x, y, z value. You enter those values, while in debug mode and run. One of your assertions fails. Now you know where the bug is. You don't want anyone else seeing that. You only want them seeing nice user friendly messages. Same with other developers... they should only have to deal with exceptions your code throws. What exceptions your code throws should be stated in the documentation so they know why it is doing that.

could you provide an example for the above design? would help in understanding much better.

When you write a class, it essentially holds on to some data. Imperitive or OO programming is all about modifying that data or state. So the idea of a class invariant is to assert that all that data is in a valid state at all times.

public class Person
{
    // Data
    private string firstName;

    private string lastName;

    private int age;

    // Methods
    public Person( string firstName, string lastName, int age )
    {
        this.firstName = firstName;

        this.lastName = lastName;

        this.age = age;

        if ( !IsValid() )
        {
            throw InvalidDataException();
        }

    }

    public string FirstName
    {
        get
        {
            Assert( IsValid() );

            return this.firstName;
        }

        set
        {
            this.firstName = value;

            Assert( IsValid() );
        }
    }

    public int Age
    {
        get
        {
            Assert( IsValid() );

            return this.age;
        }

        set
        {
            this.age = value;

            Assert( IsValid() );
        }
    }

    // Class Invariant
    private bool IsValid()
    {
        if ( this.firstName == null || this.firstName.Length == 0 )
        {
            return false;
        }

        if ( this.lastName == null || this.lastName.Length == 0 )
        {
            return false;
        }

        if ( this.age < 1 || this.age > 65 )
        {
            return false;
        }

        return true;
    }

}

This class keeps track of first/last name and age - the state. I've specified that I want the person to have a first name and last name and the age should be between 1 and 65. That's what the class invariant ( IsValid() ) is checking. In the body of "getter" methods (methods that DON'T modify the data), before doing anything, I check that the object is in a valid state. Similarly, in the body of "setter" methods (methods that DO modify the state), after doing everything, I check that the object is in a valid state.

Exceptions have the potential of getting caught in some generic try/catch block. You wouldn't know about it. The good thing about assertions is that you cannot ignore them.

Assertions!!

Regarding exceptions.... They're probably the best way to deal with errors... much better than error return codes etc... I'm saying that you shouldn't need to have try/catch blocks everywhere.

i agree totally with what you(mohn) says. even on the one gui app i've worked on there just too many try/catch blocks everywhere...


i wanted to focus on assertions actually.

The way I think about them is... assertions are for programmers, exceptions are for users. If you are coding and you have an assertion that fails, the program will halt and tell you where the error is. But you don't want the user to see that. You want an exception and a nice user friendly message.

are you trying to imply that assertions should be used only for debugging? i read this article Part 2: impact of Java's new assertion facility . i got the feeling that the author suggests the use of assertions not for debugging but for general application working. please correct me incase. and let me know how you use assertions and what do you make of this article? i understand that even though you might use assertions, eventually have to use try/catch blocks for AssertionError Exception to finally use the assertion, might increase lines of code. but do you think it will look better? conforming to the Design by Contract? being as lazy as i am i did not read on the DBC design, but sounds good. what do you feel.


Even better is to have a class invariant. That's basically a method in each class that checks that the data is in a valid state....

could you provide an example for the above design? would help in understanding much better.

Sunday, February 08, 2004

Re: Software Quality

Microsoft has brought a lot of their bad reputation as far as software quality on themselves. The Win9x line was terrible. But NT/2K and XP is quite stable. I don't think I've had a crash since I started using Win2K almost 2 years ago. Some apps crash, but you can just end the process without affecting other programs.

All the blogs and articles I read from MS devs suggest that the culture is very focused on reliablity. I am mostly exposed to this area of Microsoft - the developer bit. I don't much care for the business aspect of it. But inevitably, since they are a public company, that business case plays a role. To fix a small bug, we think, one small change... what's the problem in fixing it? But when that software is used by millions, you need to spend a lot of time/resources/money testing whether it broke something else.

I agree with your comments on security. Features came first before security. Before the internet really went big, it wasn't as critical as it is now. Just as they completely ignored the internet they seemed to also have ignored security in their products until it bit them in the arse. I think now that's also changing. Let's see what comes of their initiative.

Regarding theoretical courses... It just seems to me that the prof's do a piss poor job of getting anything through. I've taken 2 classes before and am taking one now. In each of them, I didn't get anything out of it. The point is to be able to apply that theory to practical uses. I'm clueless about that. And most people I've spoken to also feel the same. I find that it is more tuned for maths majors. In my view, if I could use those three classes to learn design basics, that would be more worthwhile. I don't plan to be a great research scientist, just to write some code.

The JUnit thing was for one project in one class. It was great because we had not been exposed to that. I actually didn't understand what it was trying to accomplish until after another student explained it to me. It wasn't anything formal.

I haven't actually read a design book. But I have scanned through some at the library. I think maybe before I make more dumb comments I should actually read one ;-) But they somehow didn't appeal too much because as you said they looked quite theoretical. And as I said before, it seems to me that it is one area where guidance is required. You need someone with prior experience to help.

Regarding exceptions... I didn't say they are inefficient (Actually technically they are very inefficient performance wise and should only be used in exceptional situations). They're probably the best way to deal with errors... much better than error return codes etc... I'm saying that you shouldn't need to have try/catch blocks everywhere.

There is a series of interviews with Anders Hejlsberg on artima.com regarding the design of C#. There was one interview focused on exceptions. This is what he said at one point...

"Surely in any kind of event-driven application like any kind of modern UI, you typically put an exception handler around your main message pump, and you just handle exceptions as they fall out that way. But you make sure you protect yourself all the way out by deallocating any resources you've grabbed, and so forth. You clean up after yourself, so you're always in a consistent state. You don't want a program where in 100 different places you handle exceptions and pop up error dialogs. What if you want to change the way you put up that dialog box? That's just terrible. The exception handling should be centralized, and you should just protect yourself as the exceptions propagate out to the handler."

I'm talking about that "centralization" of exception handling. How dyou design that for your app? I suppose the relation is to "magic" numbers vs constants. With constants you can make one change somewhere and it propogates to all areas of code. In the CDPlayer app, there are try/catch blocks everywhere in the UI. So if I need to change anything, I got to look everywhere.

BTW, check out the interview... http://www.artima.com/intv/csdes.html. It's a series - there are about 7 as of now. I think there is a similar one with James Gosling as well.

Assertions are great. That's basically what JUnit does. It's a framework for testing built around assertions. The way I think about them is... assertions are for programmers, exceptions are for users. If you are coding and you have an assertion that fails, the program will halt and tell you where the error is. But you don't want the user to see that. You want an exception and a nice user friendly message.

Definitely good prog practice. Even better is to have a class invariant. That's basically a method in each class that checks that the data is in a valid state. So, if your class keeps track of age. You know it can't ever be negative. So, check for that and if ever it is negative, return false. Then have assertions on this invariant, in all methods.

Saturday, February 07, 2004

Re: Software Quality

firstly, the watson project really sounds good. besides i have to agree that winxp has been much more stable than any windows version before. especially the ctrl+alt+del actually ends any task. i have tried it on games etc, just for fun and it is amazingly stable.

on windows, i agree that bugs can get through and that it is not possible to remove all bugs, open source or otherwise. mohn had posted a link - Bugs - on testing earlier which showed the amount of work done. the main fault with windows however is the lack of focus on security. previously they have chosen user-friendly policies/features over more secure ones. take for example the fact that you can login to an account without a password administrator or otherwise. something like this has to be removed. the security of the total system goes down. but lets see what changes take place under the trustworthy computing initiative. open source is also pretty buggy (see red hat updates), but they have placed a greater focus on security.

what i feel on uni courses..
i agree that courses can have more stuff on practical useful topics. but on the whole i feel that theoretical stuff is also very necessary. uni's have to have a fine balance between theoretical and practical. it should be majorly theoretical as in the end you should have a general idea of computers and prog in general so that you can choose/diversify your field later. but like mohn said testing, design are imp. actually design is what seperates the prog from the gurus. i guess we have reached a stage where we need to go to the next level, design. should it be taught in uni? i do not know. maybe its better learnt at a later stage after you have actually worked on some real projects and then improve your understanding of actual practical implementations and then learn some design principles.
besides i thought you had a course on testing, junit??

i think there are many books on designs. never read any, but i know they are available. have you tried any. they are higher level books requiring that you already have a good working knowledge of the language. but i think they are more on the theoretical side.

also why do you (mohn) feel that try/catch blocks are not efficient? all your code might end up being "tried and caught" but it is a very efficient system. i personally feel that error flow can be handled very effectively. if you have a specific problem/example post it so that all of us can think over it.

i am actually more of a java 1.2/1.3 prog. never really did much in the current 1.4 api. one feature i stumbled upon yesterday were assertions. there must have similar features in .net. have you guys ever used assertions? this is a block from the java 1.4 api ..

"An assertion is a statement in the JavaTM programming language that enables you to test your assumptions about your program. For example, if you write a method that calculates the speed of a particle, you might assert that the calculated speed is less than the speed of light.

Each assertion contains a boolean expression that you believe will be true when the assertion executes. If it is not true, the system will throw an error. By verifying that the boolean expression is indeed true, the assertion confirms your assumptions about the behavior of your program, increasing your confidence that the program is free of errors.

Experience has shown that writing assertions while programming is one of the quickest and most effective ways to detect and correct bugs. As an added benefit, assertions serve to document the inner workings of your program, enhancing maintainability."

what do you think of assertions? not error flow, but good prog practice?

Re: Software Quality

After reading that post, I was thinking... is there anything as complex as software that we as humans develop? I mean obviously there are a lot of things that are incredibly complex. But nothing as complex and yet as accessible. You don't have every Rahul, Hrishi and Dinesh developing rockets. But you do have them writing code and software applications.

And we (the four of us) have not yet even begun to fathom the complexity of large software systems. I still don't know how they have apps that are millions of lines of code and that actually work (most of the time ;-))! Windows is fast approaching 50 million. How the hell do they get it all to work?

Everytime a program (especially Microsoft software) hangs you're pissed... and you should be, but it's not like they've done it intentionally. There must have been a whole bunch of rigorous testing before release. But there is and probably never will be perfect software. There are so many different configurations of machines out there, it's essentially impossible to test it out on all of them.

As he's written, even I somehow feel that there is lack of education in this critical area of software development. Not enough focus is given to it. I find that the education of designing solid software is sorely lacking. The process is flawed. For example, at my University there are no classes for software design or testing. Why is that? There are a whole bunch of classes about theoretical bullshit. I know there is place for that and people who "get it" love it. But considering 90%+ of students will be writing code, not mathematical proofs, it's a shame that the syllabus isn't changed. I think it's the same everywhere. From what Avi's told me, it's not much different in the Indian system either.

Apart from learning about data-structures, computer (chip) architecture/orgranization, basics of OS and algorithms, which is all essential, the program should have a strong focus on software design and architecture. Testing is such an important part of software creation. And yet that is also ignored.

Even books do a poor job. They'll have some silly examples and won't use good practices in their code. A great example is something about exceptions. Yeah I know what execptions are and when to use them. But I don't want to have "try/catch" blocks dominating my code. I know there must be some system to follow so it's handled properly. I dunno if not looking at the right books or what.

I see all these classes for learning programming. Basically all they will teach you is the syntax. Any imbecile can pick that up. It's about design. And it should be a mandatory course in college along with a course on testing. See, the thing about this is that unlike learning a language like Java or C++, you need guidance here. You can't do it on your own. You have to learn from someone with experience and who's done this before.

Software Quality

Here's a recent blog from a Microsoft Dev

----------------------------------------------------
People who don’t build software for a living have a quite understandable attitude that you should write your program, fix all the bugs, then ship it. Why would you ever ship a product that has bugs in it? Surely that is a sign that you don’t care about quality, right?

Those of us who work on non-trivial software projects naturally see this a little differently. There are many factors that determine quality of software, and some of them are counter-intuitive. For example, the first thing to realize is that you are not trying to get rid of every bug. What you are trying to do is maximize the known quality of the product at the time you ship it. If the last bug you have in the product is that a toolbar button sometimes doesn’t display correctly, and you fix it, then you ship the product, how embarrassed will you be a little later when you find out that the code fix you took to get the toolbar to display causes your application to crash when used on machines with certain video cards? Was that the right decision? Fix every bug?

The discipline of software testing is one that is rarely taught in university, and when it is taught, it is taught in a highly theoretical way, not in the way that it needs to be done to successfully produce high quality products on a schedule demanded by commercial software.

I often interview university students applying for jobs at Microsoft. They're usually excellent people and very talented, although we have to turn away most of them. One thing that always intrigues me is when I am interviewing people who want to be a developer or tester. If I ask them to write a little code, most of them can. Then I ask them to prove to me that their code works. The responses I get to this vary so widely it is remarkable. Usually people run the example values I gave them through their algorithm and if it comes out correctly, they say it works. Then I tell them, "OK, let's say I told you this code is for use in a firewall product that is going to be shipped to millions of customers, and if any hackers get through, you get fired". Then they start to realize that they need to do a little more. So they try a few more numbers and some even try some special numbers called "boundary conditions" - the numbers on either side of a limit. They try a few more ideas. At some point they either say they're now sure, or they get stuck. So I then ask them if they think it is perfect now. Well, how do you know?

This is a central problem in large software projects. Once you get beyond about 100 lines of code, it becomes essentially impossible to prove that code is absolutely correct.

Another interesting detail here is that people constantly confuse the fact that they can’t find a problem with there being no more problems. Let's say you have a test team of one person, and a development team of 10 (at Microsoft, we have a 1:1 ratio of testers to devs, so we'd have 10 testers on that team). But what if our poor lone hypothetical tester goes on vacation for 2 weeks, and while he is away the developers fix the few bugs he has found. Is the program now bug free, just because there are no known bugs in it? It is "bug free" after all, by the definition many people would use. In fact a good friend of mine who runs his own one-man software business told me once his software is in fact bug free, since whenever any customer tells him about a bug, he fixes it. I suppose his reality is subjective. But you see the point - whether you know about the bugs or not, they're in there.

So how do you deal with them? Of course, you test test test. You use automated tests, unit tests that test code modules independently of others, integration tests to verify code works when modules are put together, genetic test algorithms that try to evolve tests that cause problems, code reviews, automated code checkers that look for known bad coding practices, etc. But all you can really say is that after awhile you are finding it hard to find bugs (or at least serious ones), which must mean you're getting it close to being acceptable to ship. Right?

One of my favorite developments in product quality that is sweeping Microsoft and now other companies as well is an effort started by a couple of colleagues of mine in Office, called simply "Watson". The idea is simple. A few hundred (or even thousands) of professional testers at Microsoft, even aided by their expert knowledge, automated tools and crack development and program management teams cannot hope to cover the stunningly diverse set of environments and activities that our actual customers have. And we hear a lot about people who have this or that weird behavior or crash on their machine that "happens all the time". Believe me, we don’t see this on our own machines. Or rather, we do at first, then we fix everything we see. In fact, the product always seems disturbingly rock solid when we ship - otherwise we wouldn’t ship it. But would every one of the 400 million Office users out there agree? Everybody has an anecdote about problems, but what are anecdotes worth? What is the true scale of the problem? Is everything random, or are there real problems shared by many people? Watson to the rescue.

As an aside, I suspect that some people will read this and say that open source magic solves this, since as the liturgy goes so many eyes are looking at the code that all the bugs are found and fixed. What you need to realize is that there are very few lawyers who can fix code bugs. And hardly any artists. Not so many high school teachers, and maybe just a handful of administrative assistants. Cathedral, bazaar, whatever. The fact is that the real user base is not part of the development loop in any significant way. With 10^8 diverse users, a base of people reporting bugs on the order of 10^4 or even 10^5, especially with a developer user profile cannot come close to discovering the issues the set of non-computer people have.

The Watson approach was simply to say: let's measure it. We'll match that 10^8 number 1 for 1. In fact, we'll go beyond measuring it, we'll categorize every crash our users have, and with their permission, collect details of the crash environment and upload those to our servers. You've probably seen that dialog that comes up asking you to report the details of your crash to Microsoft. When you report the crash, if that is a crash that someone else has already had, we increment the count on that "bucket". After a while, we'll start to get a "crash curve" histogram. On the left will be the bucket with the most "hits". On the far right will be a long list of "buckets" so rare that only one person in all those millions had that particular crash and cared to report it. This curve will then give you a "top N" for crashes. You can literally count what percentage of people would be happier if we fixed just the top 10 crashes.

When we started doing Watson, it was near the end of OfficeXP. We collected data internally for a few months, and started collecting externally for the last beta. We quickly learned that the crash curve was remarkably skewed. We saw a single bug that accounted for 27% of the crashes in one application. We also saw that many people install extra software on their machine that interferes with Office or other applications, and causes them to crash. We also saw that things we thought were fairly innocuous such as grammar checkers we licensed from vendors for some European languages were causing an unbelievable number of crashes in markets like Italy or the Netherlands. So we fixed what we could, and as a result OfficeXp was pretty stable at launch. For the first time, that was not a "gut feel" - we could measure the actual stability in the wild. For follow-on service packs for Office XP, we just attacked that crash curve and removed all the common or even mildly common crashes.

For Office2003 (and OneNote 2003), the Watson team kicked into high gear with more sophisticated debugging tools so we could more often turn those crash reports into fixes. They also started collecting more types of data (such as "hangs", when an application either goes into an infinite loop, or is taking so long to do something that it might as well be in one). We fixed so many crashing bugs and hangs that real people reported during the betas that we were way down "in the weeds", in the areas where crashes were reported only a handful of times - hard to say if that is real or just a tester trying to repro a bug they found. So again we know how stable the 2003 generation of Office applications is - we can measure it directly. What a difference from the guesswork that used to go on. In fact we know for a fact that Office 2003 at launch is already more stable than any previous release of Office ever, even after all service packs are applied, since Windows now collects Watson data for all applications.

Microsoft offers this data freely for anyone writing applications on Windows. You can find out if your shareware application is crashing or its process is being killed by your users often - sure signs of bugs you didn’t know you had. The Windows team will also give you crash dumps you can load into your debugger to see exactly what the call stack was that took you down, so you can easily fix your bugs.

This is an example of what BillG calls "Trustworthy Computing". You may think that's a bunch of hoopla, but in many ways it is quite real - things like Watson are used to directly improve the quality of our products.
----------------------------------------------------

Sunday, February 01, 2004