Hell, the FC7 server that I set up is so SLOW on the network! 1 gig RAM, 500 GB hard disks, quad core CPU. And a bummed out on board Ethernet adapter. Thats what. I tried so many things to get it to work fast. In the end, I just put in a D-Link 538Tx card and configured my system to use that as the primary interface. Sucks.
The system didn't recognize the D-Link card. I had to manually create the driver and insert it. Damn - I thought Linux was over this - extremely hard hardware configuration. But I guess it's not. This translates to more explanation of "why should I use Linux" to all the people surrounding me.
Wednesday, December 26, 2007
Tuesday, November 27, 2007
My(SQL) stupidity
What happened is this: I had a MS-Acces DB which I transported to MySQL on 'Windows'. Now, the migration toolkit decided that it will NOT preserve the case of the tables - since this is Windows. OK - I ran my School Administration System and tested it on MySQL on Windows. Worked fine. Now, I decide to move the database to my server, and here is where it all started.
First, MySQL on Linux wouldn't let me connect to it from my workstation - no privileges. Fine. I used the MySQL Administrator to add the user, grant it privileges through the Global Privileges (you got to turn the Global Privileges on in Tools->Options).
That done, I ran my program. However, this time, I got the same error - username not authenticated. I wondered and wondered why. Then it struck me like lightning. O D B C !!!
Vb.Net uses MySQL ODBC connector, to connect to the database. This System and User DSN was configured to connect to my LOCAL machine, and not the SERVER! Damn damn damn!
I changed that, and viola! Things started running! Well, almost.
Now, VB.Net starts throwing exception after exception. No table this, no table that. I'm like - you cannot be serious. I see these tables, I see data in them, then why the heck won't you see them? Another bolt of lightning. C A S E! Damn damn damn. After cursing myself and the migration toolkit, I started looking, though sadly, for this parameter to be included in my.cnf: lower_case_table_names. I set this level to 1, so that MySQL will ignore the details about case-sensitivity. Yes, I will not keep my tables case sensitive from now on. But its too much work to rename all the tables in my VB.Net code.
Anyway, its up and running like a champ - so it seems for now. I'll put up any more issues that I see from here on.
First, MySQL on Linux wouldn't let me connect to it from my workstation - no privileges. Fine. I used the MySQL Administrator to add the user, grant it privileges through the Global Privileges (you got to turn the Global Privileges on in Tools->Options).
That done, I ran my program. However, this time, I got the same error - username not authenticated. I wondered and wondered why. Then it struck me like lightning. O D B C !!!
Vb.Net uses MySQL ODBC connector, to connect to the database. This System and User DSN was configured to connect to my LOCAL machine, and not the SERVER! Damn damn damn!
I changed that, and viola! Things started running! Well, almost.
Now, VB.Net starts throwing exception after exception. No table this, no table that. I'm like - you cannot be serious. I see these tables, I see data in them, then why the heck won't you see them? Another bolt of lightning. C A S E! Damn damn damn. After cursing myself and the migration toolkit, I started looking, though sadly, for this parameter to be included in my.cnf: lower_case_table_names. I set this level to 1, so that MySQL will ignore the details about case-sensitivity. Yes, I will not keep my tables case sensitive from now on. But its too much work to rename all the tables in my VB.Net code.
Anyway, its up and running like a champ - so it seems for now. I'll put up any more issues that I see from here on.
Sunday, November 18, 2007
Moving from MS-Access to MySQL in VB.Net
I have a School Administration System , written in VB.Net, which used MS-Access. Needless to say, that was a bad decision. OK - a very bad decision. Finally, I decided to move out to MySQL. Why MySQL? Well, because my server is set up to be Linux, and I wanted all the data on all the computers to come from one place. So, first, I read up the whole of www.vbmysql.com, without which I would still be wondering about where to start from. Mike Hillyer has done an excellent job in putting that website together.
Saturday, November 17, 2007
Deleting data from all tables in MySQL
Why would I need it? Well, I am in the process of transitioning from Access to MySQL, thats why. The porting process is slow, and data keeps getting added to Access. So I figured out a way to clean old data from MySQL tables.
At the mysql prompt:
mysql> use dbName;
mysql> show tables;
**get a list of tables here**
Select the table names and paste it in your editor of choice. Add "delete from " to the beginning by running a regular expression search and replace operation for the special character ^. Thats it! You have a list of delete statements, which you can just copy paste at the MySQL prompt.
At the mysql prompt:
mysql> use dbName;
mysql> show tables;
**get a list of tables here**
Select the table names and paste it in your editor of choice. Add "delete from " to the beginning by running a regular expression search and replace operation for the special character ^. Thats it! You have a list of delete statements, which you can just copy paste at the MySQL prompt.
Sunday, November 11, 2007
AJAX and PHP
Yes, yes I know - an article on AJAX and PHP AGAIN? Well, to be frank, I think this entry will help for a beginner in understanding how to and where to begin when (s)he wants to start developing web applications.
I know that I am *quite* late in hopping on to the AJAX and PHP bandwagon. I have done a lot of Javascript and DHTML work, I have done a lot of PHP work (long long time ago in 2001). So let me set the requirements here. Steps to learning PHP and AJAX are:
1. Before you start learning about AJAX, you need to know what JavaScript and DHTML is. Start by building simple client only web pages based in JS and DHTML.
2. Once you do that, throw in some XML. This is where it gets a little complicated. XML, XSLT, XSL and who knows what. There is no end to it. Anyway, I think its best if you do some XML and some XSL and then using JS to parse the XML.
3. Now, you need to enter the world of PHP and server side scripting. You have to be very clear about how communication happens between the browser and the server, what is meant by HTTP requests and responses. Then, start by writing small PHP scripts which will communicate with the browser and show and take some data. In short, go about learning PHP the traditional way, that is, without AJAX in the mix. ( I am skipping setting up LAMP ). PHP.net has a tutorial for beginners, and so does Google.
4. Once you do that, go to the Mastering AJAX series, and read up. Its an excellent series of articles, which will help you get a grip on what AJAX is all about. Well, this is where I am currently at. I will update on more links once I do some more snooping around.
I know that I am *quite* late in hopping on to the AJAX and PHP bandwagon. I have done a lot of Javascript and DHTML work, I have done a lot of PHP work (long long time ago in 2001). So let me set the requirements here. Steps to learning PHP and AJAX are:
1. Before you start learning about AJAX, you need to know what JavaScript and DHTML is. Start by building simple client only web pages based in JS and DHTML.
2. Once you do that, throw in some XML. This is where it gets a little complicated. XML, XSLT, XSL and who knows what. There is no end to it. Anyway, I think its best if you do some XML and some XSL and then using JS to parse the XML.
3. Now, you need to enter the world of PHP and server side scripting. You have to be very clear about how communication happens between the browser and the server, what is meant by HTTP requests and responses. Then, start by writing small PHP scripts which will communicate with the browser and show and take some data. In short, go about learning PHP the traditional way, that is, without AJAX in the mix. ( I am skipping setting up LAMP ). PHP.net has a tutorial for beginners, and so does Google.
4. Once you do that, go to the Mastering AJAX series, and read up. Its an excellent series of articles, which will help you get a grip on what AJAX is all about. Well, this is where I am currently at. I will update on more links once I do some more snooping around.
Wednesday, November 7, 2007
The Future of Food
I have been thinking of this thing for a long time. Scientists working on combining various food materials to check their compatibility for the tongue and nose. PopSci, now has published an article which explains in detail, of how such things are taking place - Genes, nose, tongue, brain - the works. An excellent article for people who want to know the technicalities of good food!
Wednesday, October 24, 2007
FC7 Server for the school
Im in the process of setting up a FC7 server for my school.
Here is how I have gone about it.
Hardware: Intel Core2Duo with 1GB RAM. One 80GB SATA disk, one 160GB IDE and one 240GB IDE.
My requirements are:
1. Dovecot
2. LAMP - my database will be MySQL based. My school administration system *will be* based in PHP. My library management and store management system *will be* in PHP.
3. samba - shares, users, recycle
4. fwbackups - an excellent backup system. Really simple and easy to use. This scores over BackupPC for me.
4. mondo - I'm using this to create an image of my server.
5. cvs - I need this for my software development projects.
6. bugzilla - For software development projects, as well as for tracking van and student related changes in the school. We will follow a process for making changes in the main database. We have had tons of issues due to multiple data sources.
7. dhcp server - I have used dnsmasq for this.
Installation of FC7 was simple. Chose all the 3 configurations while installing. We are always short of machines, so the server has to be a workstation too. :( I didn't create an LVM because AFAIK mounting separate disks on another machine is a PITA (Pain In The Ass). Luckily, I went through this drill before installing FC7. If I hadn't known of the problems, I would have had had it if the system went for a toss and I had to mount those disks elsewhere. I'm sure there must be a way to mount separate disks, but I figured if it doesn't come up in the first few links of a quick Google search, then it isn't worth it!
Mount looks like this:
/dev/sdc1 on / type ext3 (rw) (80G SATA)
/dev/sdb2 on /backup type ext3 (rw) (240G)
/dev/sda2 on /school type ext3 (rw) (160G)
/dev/sda1 on /boot type ext3 (rw) (160G)
...
/dev/sdb1 is swap.
Basic post installation setup:
I created directories for our worksheets, circulars, annual day songs, a temporary scratch share, and home dirs.
0. Samba:
This is by far the most critical component of my server. If there is no Samba, there are no Windows users, and I am finished. My whole reputation is at stake in the school. I have convinced everyone about the benefits of a Linux box and blah blah blah. Samba cannot let me down!! Setup was definitely not trivial. I had to read a TON of documentation to get the permissions right. First, I created Linux users and created their own groups. For eg:
drwsrws--- 3 nkarkare nkarkare 4096 2007-11-02 22:43 nkarkare
As you see above, I set the SETUID bit for me and the group. According to the Samba Guide, doing this would ensure that the whoever connected to a share in the home dir, would write as the owner of the home dir. Unfortunately, this did not work. To get this to work, I had to use the force user = username directive. I have no idea why - probably my poor reading skills. And then I thought - why even bother with the "force user" thing. I have set up Linux groups in such a way that only certain people can access certain shares. This made me think that, heck, let me know who has created what and who has changed what. It will help me in keeping track of what is being done on the server.
I also set up recycle directories for the shares. This enables me to keep track of versions of files (it would be easier for me to set up CVS, but I can't expect all people to use it all the time). The only thing I have to do is set up a cron script to go clean the recycle directory every week or so.
Note: You HAVE to create the recycle directory before you set up the recycle facility. If you decide to keep it in the users home directory, you will need to create the directories separately.
Here is a small script which will let you do that:
(assume the homedir path is /home)
for i in `ls`
do
cd
$i mkdir .recycle
cd ..
done
Next, my task was to set up our worksheets directories. The idea was this: One main worksheets repository, and individual repositories in every persons share who deals with worksheets. the main worksheet directory will contain separate directories for every class and in that class dir, there will be separate dirs for subjects and in every subject dir, there will be dirs for unit tests. And no one, NO ONE, will be able to write anywhere except the unit test dirs. This will get some discipline into where data is being put. I don't know if there is a way in Samba to even restrict the names of the files that are made. For example, don't let a filename be less than 10 characters etc :). If there is, I would surely love to have it.
I didn't create a share for home directories. Instead, I created separate shares with /home/... as the share name [though that is not recommended for security reasons - I'm not too concerned for security as I haven't exposed the server to the Internet, and I don't expect much hacking to take place on my internal network].
I had to use the "create mask" for making sure that the files created would be 770, so that anyone in the particular group can see the files created, and can access them.
So, to summarize, here is what a home share looks like:
[nkarkare]
path = /home/nkarkare
comment = NIKHIL Share
writeable = yes
; browseable = yes
valid users = anvit, arpita, nkarkare, smp
log level = 10 vfs:10
vfs object = extd_audit recycle
recycle:repository = /tmp/.deleted/%S
recycle:keeptree = Yes
recycle:touch = Yes
recycle:versions = Yes
recycle:maxsixe = 0
recycle:exclude = *.tmp
recycle:exclude_dir = /tmp
create mask = 770
directory mask = 770
inherit permissions = Yes
And here is what my Worksheets share looks like:
[WS07]
path = /school/WS07
writeable = yes
comment = Main Worksheets Repository
; browseable = yes
valid users = anvit, arpita, nkarkare, smp
log level = 10 vfs:10
vfs object = extd_audit recycle
recycle:repository = /tmp/.deleted/WS07/
recycle:keeptree = Yes
recycle:touch = Yes
recycle:versions = Yes
recycle:maxsixe = 0
recycle:exclude = *.tmp
recycle:exclude_dir = /tmp
create mask = 770
directory mask = 770
inherit permissions = Yes
1. DHCP server:
I set up dnsmasq for DHCP services. The IP address range is 192.168.100.50 to 192.168.100.150. These are for all the machines other than a few, for which I have made entries in /etc/dnsmasq.conf. I didn't choose DHCPD because it gives me too many options, which I don't care about. Its GTK configuration tool, gdhcpd is good, but again, provides me with too many options. Dnsmasq in that regard is easy. I changed the default router to point to my ADSL modem, which filters traffic according to the IP address.
2. system-config-users:
I set up groups - one for staff and one for management. Then I created users and assigned them to these groups. I didn't create separate group ID's for each user.
3. Setting up Apache:
I changed the ServerName parameter to server.myshala.com and updated /etc/hosts accordingly. DNSMASQ seems to have automatically picked up this name update from /etc/hosts, as when I ping server.myshala.com from my workstation, I seem to get the private IP address, rather than a complaint from my ADSL gateway saying there is no server.myshala.com subdomain under our main myshala.com. Cool.
4. Mail:
I have configured sendmail with dovecot IMAP and squirrelmail. I had to make a change to sendmail.mc to make it to listen to the eth0 interface, instead of 127.0.0.1. It seems, this decision of keeping it 127.0.0.1 by default, has been taken by RedHat consciously. It resulted in "connection refused" errors when I tested telnet from my workstation to the server on port 25. I also got 0x800CCC0E errors when trying to send mail from Outlook Express via sendmail - which led me to check whether I could just simply telnet to port 25 of my server.
I have consciously decided not to have POP as it will result in all mail being downloaded on workstations. I want to keep the mails on the server.
5. Bugzilla:
I installed from yum, ran checksetup.pl, which complained about me not having made changes to localconfig. I changed $db_pass, and went to MySQL to set up the bugs user:
mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES,
CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.*
TO bugs@localhost IDENTIFIED BY 'myPassword';
mysql> FLUSH PRIVILEGES;
I ran checksetup.pl again and it was as smooth as wine thereafter. I am going to create two products there - Millennium and RudiMend. Millennium will have all school related bug tracking changes - like van change, address change, etc. RudiMend will have software specific stuff - like the educational software we have, or the school administration system that we use etc.
6. Fwbackups:
There was nothing special I did here. I just set up 2 back up policies, to be made on a separate IDE disk. One for the home dirs, and another for my school related stuff. I will manage CVS backup, and MySQL backup later on.
7. Mondo: I found Mondo to be more usable than G4L, which seems to be more popular. I am planning to make an image of my server before anyone puts any data on it. Once I create an image of my hard drive, it will be easy for me to restore it if anything goes wrong. Mondo has the following dependencies: afio,buffer,mindi,mindi-busybox,syslinux - other than syslinux, all of them are available on the Mondo FTP site. I upgraded syslinux through yum.
With that, it looks like my server is set up and ready to go! It was quite easy to set up, though Samba took some time - but no regrets there, as it IS the MOST vital component of my server. I get a lot of vomit from the samba logs, but that is good, as I know what is happening at any point of time.
Here is how I have gone about it.
Hardware: Intel Core2Duo with 1GB RAM. One 80GB SATA disk, one 160GB IDE and one 240GB IDE.
My requirements are:
1. Dovecot
2. LAMP - my database will be MySQL based. My school administration system *will be* based in PHP. My library management and store management system *will be* in PHP.
3. samba - shares, users, recycle
4. fwbackups - an excellent backup system. Really simple and easy to use. This scores over BackupPC for me.
4. mondo - I'm using this to create an image of my server.
5. cvs - I need this for my software development projects.
6. bugzilla - For software development projects, as well as for tracking van and student related changes in the school. We will follow a process for making changes in the main database. We have had tons of issues due to multiple data sources.
7. dhcp server - I have used dnsmasq for this.
Installation of FC7 was simple. Chose all the 3 configurations while installing. We are always short of machines, so the server has to be a workstation too. :( I didn't create an LVM because AFAIK mounting separate disks on another machine is a PITA (Pain In The Ass). Luckily, I went through this drill before installing FC7. If I hadn't known of the problems, I would have had had it if the system went for a toss and I had to mount those disks elsewhere. I'm sure there must be a way to mount separate disks, but I figured if it doesn't come up in the first few links of a quick Google search, then it isn't worth it!
Mount looks like this:
/dev/sdc1 on / type ext3 (rw) (80G SATA)
/dev/sdb2 on /backup type ext3 (rw) (240G)
/dev/sda2 on /school type ext3 (rw) (160G)
/dev/sda1 on /boot type ext3 (rw) (160G)
...
/dev/sdb1 is swap.
Basic post installation setup:
I created directories for our worksheets, circulars, annual day songs, a temporary scratch share, and home dirs.
0. Samba:
This is by far the most critical component of my server. If there is no Samba, there are no Windows users, and I am finished. My whole reputation is at stake in the school. I have convinced everyone about the benefits of a Linux box and blah blah blah. Samba cannot let me down!! Setup was definitely not trivial. I had to read a TON of documentation to get the permissions right. First, I created Linux users and created their own groups. For eg:
drwsrws--- 3 nkarkare nkarkare 4096 2007-11-02 22:43 nkarkare
As you see above, I set the SETUID bit for me and the group. According to the Samba Guide, doing this would ensure that the whoever connected to a share in the home dir, would write as the owner of the home dir. Unfortunately, this did not work. To get this to work, I had to use the force user = username directive. I have no idea why - probably my poor reading skills. And then I thought - why even bother with the "force user" thing. I have set up Linux groups in such a way that only certain people can access certain shares. This made me think that, heck, let me know who has created what and who has changed what. It will help me in keeping track of what is being done on the server.
I also set up recycle directories for the shares. This enables me to keep track of versions of files (it would be easier for me to set up CVS, but I can't expect all people to use it all the time). The only thing I have to do is set up a cron script to go clean the recycle directory every week or so.
Note: You HAVE to create the recycle directory before you set up the recycle facility. If you decide to keep it in the users home directory, you will need to create the directories separately.
Here is a small script which will let you do that:
(assume the homedir path is /home)
for i in `ls`
do
cd
$i mkdir .recycle
cd ..
done
Next, my task was to set up our worksheets directories. The idea was this: One main worksheets repository, and individual repositories in every persons share who deals with worksheets. the main worksheet directory will contain separate directories for every class and in that class dir, there will be separate dirs for subjects and in every subject dir, there will be dirs for unit tests. And no one, NO ONE, will be able to write anywhere except the unit test dirs. This will get some discipline into where data is being put. I don't know if there is a way in Samba to even restrict the names of the files that are made. For example, don't let a filename be less than 10 characters etc :). If there is, I would surely love to have it.
I didn't create a share for home directories. Instead, I created separate shares with /home/... as the share name [though that is not recommended for security reasons - I'm not too concerned for security as I haven't exposed the server to the Internet, and I don't expect much hacking to take place on my internal network].
I had to use the "create mask" for making sure that the files created would be 770, so that anyone in the particular group can see the files created, and can access them.
So, to summarize, here is what a home share looks like:
[nkarkare]
path = /home/nkarkare
comment = NIKHIL Share
writeable = yes
; browseable = yes
valid users = anvit, arpita, nkarkare, smp
log level = 10 vfs:10
vfs object = extd_audit recycle
recycle:repository = /tmp/.deleted/%S
recycle:keeptree = Yes
recycle:touch = Yes
recycle:versions = Yes
recycle:maxsixe = 0
recycle:exclude = *.tmp
recycle:exclude_dir = /tmp
create mask = 770
directory mask = 770
inherit permissions = Yes
And here is what my Worksheets share looks like:
[WS07]
path = /school/WS07
writeable = yes
comment = Main Worksheets Repository
; browseable = yes
valid users = anvit, arpita, nkarkare, smp
log level = 10 vfs:10
vfs object = extd_audit recycle
recycle:repository = /tmp/.deleted/WS07/
recycle:keeptree = Yes
recycle:touch = Yes
recycle:versions = Yes
recycle:maxsixe = 0
recycle:exclude = *.tmp
recycle:exclude_dir = /tmp
create mask = 770
directory mask = 770
inherit permissions = Yes
1. DHCP server:
I set up dnsmasq for DHCP services. The IP address range is 192.168.100.50 to 192.168.100.150. These are for all the machines other than a few, for which I have made entries in /etc/dnsmasq.conf. I didn't choose DHCPD because it gives me too many options, which I don't care about. Its GTK configuration tool, gdhcpd is good, but again, provides me with too many options. Dnsmasq in that regard is easy. I changed the default router to point to my ADSL modem, which filters traffic according to the IP address.
2. system-config-users:
I set up groups - one for staff and one for management. Then I created users and assigned them to these groups. I didn't create separate group ID's for each user.
3. Setting up Apache:
I changed the ServerName parameter to server.myshala.com and updated /etc/hosts accordingly. DNSMASQ seems to have automatically picked up this name update from /etc/hosts, as when I ping server.myshala.com from my workstation, I seem to get the private IP address, rather than a complaint from my ADSL gateway saying there is no server.myshala.com subdomain under our main myshala.com. Cool.
4. Mail:
I have configured sendmail with dovecot IMAP and squirrelmail. I had to make a change to sendmail.mc to make it to listen to the eth0 interface, instead of 127.0.0.1. It seems, this decision of keeping it 127.0.0.1 by default, has been taken by RedHat consciously. It resulted in "connection refused" errors when I tested telnet from my workstation to the server on port 25. I also got 0x800CCC0E errors when trying to send mail from Outlook Express via sendmail - which led me to check whether I could just simply telnet to port 25 of my server.
I have consciously decided not to have POP as it will result in all mail being downloaded on workstations. I want to keep the mails on the server.
5. Bugzilla:
I installed from yum, ran checksetup.pl, which complained about me not having made changes to localconfig. I changed $db_pass, and went to MySQL to set up the bugs user:
mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES,
CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.*
TO bugs@localhost IDENTIFIED BY 'myPassword';
mysql> FLUSH PRIVILEGES;
I ran checksetup.pl again and it was as smooth as wine thereafter. I am going to create two products there - Millennium and RudiMend. Millennium will have all school related bug tracking changes - like van change, address change, etc. RudiMend will have software specific stuff - like the educational software we have, or the school administration system that we use etc.
6. Fwbackups:
There was nothing special I did here. I just set up 2 back up policies, to be made on a separate IDE disk. One for the home dirs, and another for my school related stuff. I will manage CVS backup, and MySQL backup later on.
7. Mondo: I found Mondo to be more usable than G4L, which seems to be more popular. I am planning to make an image of my server before anyone puts any data on it. Once I create an image of my hard drive, it will be easy for me to restore it if anything goes wrong. Mondo has the following dependencies: afio,buffer,mindi,mindi-busybox,syslinux - other than syslinux, all of them are available on the Mondo FTP site. I upgraded syslinux through yum.
With that, it looks like my server is set up and ready to go! It was quite easy to set up, though Samba took some time - but no regrets there, as it IS the MOST vital component of my server. I get a lot of vomit from the samba logs, but that is good, as I know what is happening at any point of time.
Subscribe to:
Comments (Atom)