public class FileGrabber
extends java.lang.Object
FileGrabber grabber = new FileGrabber("data", "http://www.fileserver.com");
grabber.get("localFile", "remoteFile");
By default, the FileGrabber will simply return the localFile if it exists. To
force updates from a remote location, you can set options:
int options = FileGrabber.CHECK_HASH;
grabber.get("localFile", "remoteFile", options);
The CHECK_HASH flag downloads the remote file if its sha1 hash differs from
that of your local copy. There is also a FORCE_REMOTE flag that forces the
remote file to be downloaded always.
boolean download = true;
File dest = new File("localFile");
URI source = new URI("remoteFile");
File local = grabber.getLocal(dest);
if (local != null) {
boolean match = false;
try {
match = grabber.equalsHash(dest, source);
} catch (FileGrabberException e) {
System.out.println(e.getMessage());
}
download = !match;
}
if (download) {
try {
local = grabber.getRemote(dest, source);
} catch (FileGrabberException e) {
System.out.println(e.getMessage());
}
}
if (local == null) {
throw new RuntimeException("Unable to get file.");
}
See VFS2 File
Systems for further details regarding supported remote filesystems and
URI syntax.Modifier and Type | Field and Description |
---|---|
static int |
CHECK_HASH
Check file hashes, and if different, get from remote
|
static int |
DEFAULT_LOG_LEVEL |
static Logger |
DEFAULT_LOGGER |
static int |
DEFAULT_OPTIONS |
static int |
DOWNLOAD_ZIP
If file is in a remote zip file, get a local copy
of the entire zip file first
|
static int |
FORCE_REMOTE
Always read from remote if possible
|
int |
myOptions |
Constructor and Description |
---|
FileGrabber()
Default constructor, sets the local directory to the current path, and
sets the default URI to be empty.
|
FileGrabber(java.io.File downloadDir,
URIx remoteSource)
Sets default paths
|
FileGrabber(java.lang.String downloadPath)
Sets default download directory, leaves source as null
|
FileGrabber(java.lang.String downloadPath,
java.lang.String remoteSourceName)
Sets local download path and remote URI source by parsing the supplied
strings
|
Modifier and Type | Method and Description |
---|---|
void |
addIdentityRepository(URIxMatcher matcher,
SimpleIdentityRepository repo)
Adds an identity repository consisting of a set of private RSA keys for
use with SFTP authentication.
|
void |
addTransferListener(FileTransferListener listener)
Adds a FileTransferListener object that responds to transfer events.
|
void |
addUserAuthenticator(URIxMatcher matcher,
org.apache.commons.vfs2.UserAuthenticator auth)
Adds an authenticator for HTTPS, SFTP, that can respond to
domain/username/password requests.
|
void |
closeStreams()
Must be called to free resources after streams have been read.
|
boolean |
equalsHash(java.io.File file,
URIx uri)
Compares sha1 hash values between a local file and remote URI
|
boolean |
equalsHash(java.lang.String relPath) |
java.io.File |
get(java.io.File path,
URIx source)
Downloads a file using the default options
|
java.io.File |
get(java.io.File dest,
URIx source,
int options)
Returns a file handle to a local version of the requested file.
|
java.io.File |
get(java.lang.String sourceName)
Downloads a file using same relative path for source and destination
and default options
|
java.io.File |
get(java.lang.String sourceName,
int options)
Downloads a file using same relative path for source and destination
|
java.io.File |
get(java.lang.String destName,
java.lang.String sourceName)
Downloads a file with default options
|
java.io.File |
get(java.lang.String destName,
java.lang.String sourceName,
int options)
Converts the supplied destination path and source URI to a File and URI
object, respectively, and downloads the remote file according to the
supplied options.
|
java.io.File |
getAbsoluteFile(java.io.File relFile)
Converts a relative file to an absolute one using the object's download
directory.
|
java.io.File |
getAbsoluteFile(java.lang.String relPath)
Converts a relative file to an absolute one using the object's download
directory.
|
URIx |
getAbsoluteURI(java.lang.String relURIstr)
Converts a relative URI to an absolute one, using the remoteSource as a
base.
|
URIx |
getAbsoluteURI(URIx relURI)
Converts a relative URI to an absolute one, using the remote source as a
base.
|
boolean |
getConsoleProgressPrinting()
Returns true if default printing of file transfer progress to the
console is enabled.
|
java.io.File |
getDownloadDir() |
java.io.InputStream |
getInputStream(java.io.File localCopy,
URIx source,
int options)
Returns an input stream for the file located at either
localCopy
or source (according to options). |
java.io.InputStream |
getInputStream(java.lang.String sourceName) |
java.io.InputStream |
getInputStream(java.lang.String sourceName,
int options) |
java.io.InputStream |
getInputStream(java.lang.String localCopy,
java.lang.String sourceName) |
java.io.InputStream |
getInputStream(java.lang.String localCopy,
java.lang.String sourceName,
int options) |
java.io.InputStream |
getInputStream(URIx source) |
java.io.InputStream |
getInputStream(URIx source,
int options) |
java.io.File |
getLastFile()
Gets the last file retrieved.
|
java.io.File |
getLocal(java.io.File file)
Retrieves a local file if it exists, null otherwise.
|
java.io.File |
getLocal(java.lang.String fileName)
Retrieves a local file if it exists, null otherwise.
|
java.lang.String |
getLocalHash(java.io.File file)
Gets the sha1 hash of a local file
|
java.lang.String |
getLocalHash(java.lang.String fileName)
Gets the sha1 hash of a local file
|
static Logger |
getLogger()
Gets the logger for printing messages
|
int |
getOptions() |
java.io.File |
getRemote(java.io.File dest,
URIx source)
Retrieves a remote file if it exists, null otherwise.
|
java.io.File |
getRemote(java.lang.String sourceName)
getRemote(String, String) with dest=null |
java.io.File |
getRemote(java.lang.String destName,
java.lang.String sourceName)
Retrieves a remote file if it exists, null otherwise.
|
java.io.File |
getRemote(URIx source)
getRemote(File, URIx) with dest=null |
java.lang.String |
getRemoteHash(java.lang.String uriStr)
Fetches a hash file from a remote location.
|
java.lang.String |
getRemoteHash(URIx uri)
Fetches a hash file from a remote location.
|
URIx |
getRemoteSource() |
static FileGrabber |
getStaticGrabber()
Returns a FileGrabber object used for
staticGet(File, URIx) . |
FileTransferListener[] |
getTransferListeners()
Gets the set of listeners for all transfers handled by this FileGrabber
object
|
FileTransferMonitor |
getTransferMonitor()
Returns the FileTransferMonitor object that is is responsible for
detecting changes to the destination file and firing FileTransferEvents to
the set of
FileTransferListener s. |
void |
removeTransferListener(FileTransferListener listener)
Removes a listener that is listener to transfer events.
|
void |
setConsoleProgressPrinting(boolean enable)
Enables or disables default printing of file transfer progress to the
console.
|
void |
setDownloadDir(java.io.File dir)
Sets directory where files are downloaded
|
void |
setDownloadDir(java.lang.String path)
Sets directory where files are downloaded
|
static void |
setLogger(Logger log)
Sets the logger for printing messages, defaults
to printing to console.
|
void |
setMonitorSleep(double seconds)
Sets the sleep time between checking for transfer updates.
|
void |
setOptions(int options) |
void |
setRemoteSource(java.lang.String uriStr)
Sets the base URI for remote files, this is attached to any relative URIs
provided in the get(...) methods
|
void |
setRemoteSource(URIx uri)
Sets the base URI for remote files, this is attached to any relative URIs
provided in the get(...) methods
|
static void |
setStaticGrabber(FileGrabber grabber)
Sets the FileGrabber used for
staticGet(File, URIx) . |
void |
setTransferMonitor(FileTransferMonitor monitor)
Sets the FileTransferMonitor which is responsible for detecting when the
destination file has changed in order to fire update events.
|
void |
setVerbosityLevel(int level)
Sets the verbosity level of this FileGrabber.
|
void |
setVerbosityLevel(Logger.LogLevel level)
Sets the verbosity level of this FileGrabber.
|
static java.io.File |
staticGet(java.io.File dest,
URIx source)
Static convenience method for downloading a file.
|
static java.io.File |
staticGet(java.io.File dest,
URIx source,
int options)
Static convenience method for downloading a file in a single shot.
|
static java.io.File |
staticGet(java.lang.String destName,
java.lang.String sourceName)
Static convenience method for downloading a file with default options.
|
static java.io.File |
staticGet(java.lang.String destName,
java.lang.String sourceName,
int options)
Static convenience method for downloading a file.
|
boolean |
wasLastRemote()
Returns true if the last file was fetched from remote,
false if last file was a local copy
|
public static final int FORCE_REMOTE
public static final int CHECK_HASH
public static final int DOWNLOAD_ZIP
public static int DEFAULT_OPTIONS
public static int DEFAULT_LOG_LEVEL
public static Logger DEFAULT_LOGGER
public int myOptions
public FileGrabber()
public FileGrabber(java.io.File downloadDir, URIx remoteSource)
downloadDir
- the local path to save files toremoteSource
- the remote base URI to download files frompublic FileGrabber(java.lang.String downloadPath)
downloadPath
- the local path to save files topublic FileGrabber(java.lang.String downloadPath, java.lang.String remoteSourceName)
downloadPath
- the local path to save files toremoteSourceName
- the remote base URIpublic void setDownloadDir(java.io.File dir)
dir
- default download directorypublic void setDownloadDir(java.lang.String path)
path
- default download directorypublic java.io.File getDownloadDir()
public void setRemoteSource(URIx uri)
uri
- public void setRemoteSource(java.lang.String uriStr) throws URIxSyntaxException
uriStr
- URIxSyntaxException
- if uriStr is malformedpublic URIx getRemoteSource()
public URIx getAbsoluteURI(java.lang.String relURIstr) throws URIxSyntaxException
relURIstr
- the relative URIURIxSyntaxException
- if URI string is malformedpublic URIx getAbsoluteURI(URIx relURI)
relURI
- public java.io.File getAbsoluteFile(java.io.File relFile)
relFile
- the relative filepublic java.io.File getAbsoluteFile(java.lang.String relPath)
relPath
- public java.lang.String getRemoteHash(java.lang.String uriStr) throws FileGrabberException, URIxSyntaxException
uriStr
- the URI of the file to obtain the hashFileGrabberException
- if cannot retrieve remote hashURIxSyntaxException
- if uriStr is malformedpublic java.lang.String getRemoteHash(URIx uri) throws FileGrabberException
uri
- the URI of the file to obtain the hashFileGrabberException
public java.lang.String getLocalHash(java.lang.String fileName)
fileName
- file to compute the hash ofFileGrabberException
- if fails to generate hash of local filepublic java.lang.String getLocalHash(java.io.File file)
file
- to compute the hash ofFileGrabberException
- if fails to generate hash of local filepublic boolean equalsHash(java.io.File file, URIx uri) throws FileGrabberException
file
- local file of which to compute hashuri
- remote file of which to determine hashFileGrabberException
- if can't get either hashpublic boolean equalsHash(java.lang.String relPath) throws FileGrabberException
FileGrabberException
public java.io.File getLocal(java.io.File file)
file
- path for the local filepublic java.io.File getLocal(java.lang.String fileName)
fileName
- public java.io.File getRemote(java.io.File dest, URIx source) throws FileGrabberException
dest
- the destination file (local)source
- the source URIFileGrabberException
- if downloading the remote file failspublic java.io.File getRemote(java.lang.String destName, java.lang.String sourceName) throws FileGrabberException, URIxSyntaxException
destName
- the destination file (local)sourceName
- the source URIURIxSyntaxException
- if the source URI is malformedFileGrabberException
- if grabbing the remote file failspublic java.io.File getRemote(java.lang.String sourceName) throws FileGrabberException
getRemote(String, String)
with dest=nullFileGrabberException
public java.io.File getRemote(URIx source) throws FileGrabberException
getRemote(File, URIx)
with dest=nullFileGrabberException
public java.io.File get(java.io.File dest, URIx source, int options) throws FileGrabberException
dest
- the local path (relative or absolute) to download file tosource
- the remote URI to cacheoptions
- set of options, either FORCE_REMOTE or CHECK_HASHFileGrabberException
- only if there is no local copy of the file
at the end of the function callpublic java.io.File get(java.io.File path, URIx source) throws FileGrabberException
FileGrabberException
get(File, URIx, int)
public java.io.File get(java.lang.String sourceName, int options) throws FileGrabberException
FileGrabberException
get(String, String, int)
public java.io.File get(java.lang.String sourceName) throws FileGrabberException
FileGrabberException
get(String, String, int)
public java.io.File get(java.lang.String destName, java.lang.String sourceName, int options) throws URIxSyntaxException, FileGrabberException
URIxSyntaxException
- if the sourceName is malformedFileGrabberException
- if download fails.get(File, URIx, int)
public java.io.File get(java.lang.String destName, java.lang.String sourceName) throws FileGrabberException
FileGrabberException
get(String, String, int)
public static void setLogger(Logger log)
log
- public static Logger getLogger()
public void setVerbosityLevel(int level)
level
- between 0 (off) and 6 (print everything).public void setVerbosityLevel(Logger.LogLevel level)
level
- The minimum message level to print, TRACE printing everything, NONE
printing nothing.public void setConsoleProgressPrinting(boolean enable)
getTransferListeners()
.enable
- enables or disables console progress printingpublic boolean getConsoleProgressPrinting()
public void addTransferListener(FileTransferListener listener)
listener
- The file listener objectpublic void removeTransferListener(FileTransferListener listener)
listener
- FileTransferListener to removepublic FileTransferListener[] getTransferListeners()
public FileTransferMonitor getTransferMonitor()
FileTransferListener
s. By default, a
MultiFileTransferMonitor
is created which can monitor several file
transfers at once.addTransferListener(FileTransferListener)
public void setTransferMonitor(FileTransferMonitor monitor)
monitor
- the monitor to usepublic void setMonitorSleep(double seconds)
FileTransferMonitor
object that runs in a separate
thread. This periodically polls the destination files to see if the
transfer has progressed, and fires events to any FileTransferListeners
supplied by the addTransferListener(FileTransferListener)
function.seconds
- public void addUserAuthenticator(URIxMatcher matcher, org.apache.commons.vfs2.UserAuthenticator auth)
matcher
- object that checks whether a supplied URI matches a given set of criteriaauth
- the authenticator objectpublic void addIdentityRepository(URIxMatcher matcher, SimpleIdentityRepository repo)
matcher
- object that checks whether supplied URI matches a given set of criteriarepo
- repository containing a set of private RSA keyspublic static FileGrabber getStaticGrabber()
staticGet(File, URIx)
.
Creates one if it does not yet exist.public static void setStaticGrabber(FileGrabber grabber)
staticGet(File, URIx)
.grabber
- the FileGrabber to set for static operationspublic static java.io.File staticGet(java.io.File dest, URIx source, int options) throws FileGrabberException
FileGrabberException
get(File, URIx, int)
public static java.io.File staticGet(java.io.File dest, URIx source) throws FileGrabberException
FileGrabberException
staticGet(File, URIx, int)
public static java.io.File staticGet(java.lang.String destName, java.lang.String sourceName, int options) throws FileGrabberException
FileGrabberException
get(String, String, int)
public static java.io.File staticGet(java.lang.String destName, java.lang.String sourceName) throws FileGrabberException
FileGrabberException
get(String, String, int)
public void setOptions(int options)
public int getOptions()
public java.io.InputStream getInputStream(java.io.File localCopy, URIx source, int options)
localCopy
or source
(according to options). Works with absolute paths and
source URIs, otherwise combines path and source URI with the default
download directory and remote source uri, respectively. If the
destination is null or a directory, then the filename of source is
appended.
If there is any internal problem, (such as failing to obtain a hash, or
failing to download a file), the function will log the error message and
continue.localCopy
- the local path (relative or absolute) to check for overriding filesource
- the remote URI to read fromoptions
- set of options, from FORCE_REMOTE, DOWNLOAD_ZIP, CHECK_HASH
FileGrabberException
- if we cannot open the streampublic java.io.InputStream getInputStream(URIx source, int options) throws FileGrabberException
FileGrabberException
public java.io.InputStream getInputStream(URIx source)
public java.io.InputStream getInputStream(java.lang.String localCopy, java.lang.String sourceName, int options) throws FileGrabberException
FileGrabberException
public java.io.InputStream getInputStream(java.lang.String localCopy, java.lang.String sourceName) throws FileGrabberException
FileGrabberException
public java.io.InputStream getInputStream(java.lang.String sourceName, int options) throws FileGrabberException
FileGrabberException
public java.io.InputStream getInputStream(java.lang.String sourceName) throws FileGrabberException
FileGrabberException
public void closeStreams()
public java.io.File getLastFile()
public boolean wasLastRemote()