[#SFOS-1119] add more checks to DfsPathExists

SmartFrog

add more checks to DfsPathExists

Created: 17/Feb/09 01:25 PM (GMT) Updated: 17/Feb/09 05:34 PM (GMT)

Component/s:

_service_hadoop

Affects Version/s:

3.17.010

Fix Version/s:

3.17.010

Time Tracking:

Not Specified

Compatibility:

backwards compatible

Description

« Hide

Add more checks to DfsPathExists

* for a directory, the min and max num of files
* for a directory, the min and max total file size, if all bytes are added up

This makes it easier to test that things are being set up for job submission

All

Comments

Work Log

Change History

Sort Order:

[ Permlink | « Hide ]

Steve Loughran added a comment - 17/Feb/09 02:54 PM (GMT)

some problems here, filesize is returning blocksize, not actual file length

Caused by: SmartFrogLifecycleException:: [sfStart] Failed to start a new child 'assertFileExists'., cause: SmartFrogLivenessException:: File /test-filename in DFS[DFSClient[clientName=DFSClient_1073481522, ugi=slo,users,adm,dialout,fax,cdrom,floppy,tape,audio,dip,plugdev,scanner,admin,fuse,sambashare]] is too big at 67108864 bytes for the maximum size 40, SmartFrog 3.17.005dev (2009-02-13 11:47:21 GMT), SmartFrog 3.17.005dev (2009-02-13 11:47:21 GMT), primSFCompleteName: HOST morzine.hpl.hp.com:rootProcess:testClusteredFilesystem:tests, primContext: included, reference: HOST morzine.hpl.hp.com:rootProcess:testClusteredFilesystem:tests, primContext: included
at org.smartfrog.sfcore.common.SmartFrogLifecycleException.forward(SmartFrogLifecycleException.java:232)
at org.smartfrog.sfcore.common.SmartFrogLifecycleException.sfStart(SmartFrogLifecycleException.java:126)
at org.smartfrog.sfcore.compound.CompoundImpl.sfCreateNewChild(CompoundImpl.java:389)
Caused by: SmartFrogLivenessException:: File /test-filename in DFS[DFSClient[clientName=DFSClient_1073481522, ugi=slo,users,adm,dialout,fax,cdrom,floppy,tape,audio,dip,plugdev,scanner,admin,fuse,sambashare]] is too big at 67108864 bytes for the maximum size 40, SmartFrog 3.17.005dev (2009-02-13 11:47:21 GMT)
at org.smartfrog.services.hadoop.components.dfs.DfsPathExistsImpl.checkPathExists(DfsPathExistsImpl.java:246)
at org.smartfrog.services.hadoop.components.dfs.DfsPathExistsImpl.sfStart(DfsPathExistsImpl.java:89)
at org.smartfrog.sfcore.compound.CompoundImpl.sfCreateNewChild(CompoundImpl.java:383)

[ Permlink | « Hide ]

Steve Loughran added a comment - 17/Feb/09 02:56 PM (GMT)

could check file permissions too?

[ Permlink | « Hide ]

Steve Loughran added a comment - 17/Feb/09 05:34 PM (GMT)

done!