Linux filesystem question?

Question

Not exactly a beginner, but I need help none the less. I am a webmaster, both designing and hosting my client's sites (and please don't comment on the hosting I am Net+ certified). I have written my own set of PHP libraries for dynamic pages, but before space becomes an issue(despite the raided 2TBs of storage I use very little-per-client thanks to my php lib), I need to know what takes up more space: many, sometimes up to 100 very tiny files, one flat file with added regex filters (such as a ":" to seperate entries which would be in the smaller files), or just a mysql database? I know the smaller files take up a little more room thanks to iNodes, but that shouldn't make a difference. By the way this is all on a 1TB RAID 5 drive utilizing the ext3 file system.

Answer 1

Most modern filesystems will use less space for one large file than a large number of small files of the same combined size, because of overhead required for each file and blocks that are not 100% full.



I did quick experiment on ext3 filesystem to confirm this - 1000 files of just few bytes each took about 4 mb off filesystem (3mb overhead), where one 1mb file took off 1028 kb (4kb overhead).



As far as MySQL goes, there's no quick answer to that - it highly depends on how you organize data and indexes - but even un-indexed data table will likely will take more space than equivalent text file.



There is a trade-off between speed of access and space usage. Data taking less space is not necessarily more easily accessible. That's the difference between OLTP and data warehouse systems.

Answer 2

The smaller files are the most wasteful. Mysql is very economical in storage, and larger files are quite good, but I find they slow the system. I am hosting on a 500GB raid 5 system, I keep all the client files apart, different folder per site where many php files are duplicated, as this increases individual security. I do share the database between some sites, but each has its' own database for site user and file security. I have other things on the machine, and am unlikely to run short of file space for about 5 years.

Answer 3

the latest variations of the two Fedora and Ubuntu use EXT4 because of the fact the default filesystem. that's what i desire to advise too. EXT4 is plenty swifter than EXT3 in the two fsck besides as startup/shutdown cases. i'm looking forward to btrfs yet its nonetheless some 3 hundred and sixty 5 days away.