Class TwitterProxyReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.core.ProxyReader
com.northconcepts.datapipeline.internal.lang.IntegrationProxyReader
com.northconcepts.datapipeline.twitter.TwitterProxyReader
- Direct Known Subclasses:
TwitterUserLookupReader
Base class for all Twitter ProxyReaders.
-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsFields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionTwitterProxyReader(String rateLimitResource, String rateLimitKey, TwitterProvider provider, DataReader nestedDataReader) TwitterProxyReader(String rateLimitResource, String rateLimitKey, String consumerKey, String consumerSecret, DataReader nestedDataReader) TwitterProxyReader(String rateLimitResource, String rateLimitKey, String consumerKey, String consumerSecret, String accessToken, String accessTokenSecret, DataReader nestedDataReader) -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.Returns the action taken when the API call limit is reached (defaults toApiLimitPolicy.FAIL).Returns the converter that copies Twitter objects into records (defaults toITwitterConverter.DEFAULT).Returns how multi-valued entity fields are stored (defaults toEntityExpansionPolicy.TO_STRING).final TwitterRateLimitReturns the cached rate limit, fetching it from the API only if none has been loaded.final TwitterRateLimitgetRateLimit(boolean autoReload) Returns the rate limit, reloading it if none is cached or ifautoReloadis true and it has reset.booleanprotected booleanMarks the rate limit as exceeded and returns the API limit policy's answer: true to retry, false to stop reading.voidopen()Makes this endpoint ready for reading or writing.final voidFetches the current rate limit from the API, replacing the cached one.setApiLimitPolicy(IApiLimitPolicy apiLimitPolicy) Sets the action taken when the API call limit is reached (defaults toApiLimitPolicy.FAIL).setConverter(ITwitterConverter converter) Sets the converter that copies Twitter objects into records (defaults toITwitterConverter.DEFAULT).setEntityExpansionPolicy(IEntityExpansionPolicy entityExpansionPolicy) Sets how multi-valued entity fields are stored (defaults toEntityExpansionPolicy.TO_STRING).setRateLimit(RateLimitStatus rateLimitStatus) Replaces the cached rate limit; null clears it so the nextgetRateLimit()call reloads it from the API.setRateLimit(TwitterResponse response) Updates the rate limit from the response's status; does nothing if the response or its status is null.protected TwitterProxyReadersetRateLimitExceeded(boolean rateLimitExceeded) Methods inherited from class com.northconcepts.datapipeline.core.ProxyReader
available, close, getNestedReader, interceptRecord, map, map, readImpl, setNestedDataReader, setNestedDataReaderMethods inherited from class com.northconcepts.datapipeline.core.DataReader
addLineage, getBufferSize, getNestedEndpoint, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
EOF
-
-
Constructor Details
-
TwitterProxyReader
public TwitterProxyReader(String rateLimitResource, String rateLimitKey, TwitterProvider provider, DataReader nestedDataReader) -
TwitterProxyReader
public TwitterProxyReader(String rateLimitResource, String rateLimitKey, String consumerKey, String consumerSecret, DataReader nestedDataReader) -
TwitterProxyReader
-
-
Method Details
-
open
Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classIntegrationProxyReader- Throws:
DataException
-
getProvider
-
getApiLimitPolicy
Returns the action taken when the API call limit is reached (defaults toApiLimitPolicy.FAIL). -
setApiLimitPolicy
Sets the action taken when the API call limit is reached (defaults toApiLimitPolicy.FAIL). -
getRateLimit
Returns the cached rate limit, fetching it from the API only if none has been loaded. -
onRateLimitExceeded
protected boolean onRateLimitExceeded()Marks the rate limit as exceeded and returns the API limit policy's answer: true to retry, false to stop reading. -
setRateLimitExceeded
-
isRateLimitExceeded
public boolean isRateLimitExceeded() -
getRateLimit
Returns the rate limit, reloading it if none is cached or ifautoReloadis true and it has reset. -
setRateLimit
Updates the rate limit from the response's status; does nothing if the response or its status is null. -
setRateLimit
Replaces the cached rate limit; null clears it so the nextgetRateLimit()call reloads it from the API. -
reloadRateLimit
public final void reloadRateLimit()Fetches the current rate limit from the API, replacing the cached one. -
getEntityExpansionPolicy
Returns how multi-valued entity fields are stored (defaults toEntityExpansionPolicy.TO_STRING). -
setEntityExpansionPolicy
Sets how multi-valued entity fields are stored (defaults toEntityExpansionPolicy.TO_STRING). -
getConverter
Returns the converter that copies Twitter objects into records (defaults toITwitterConverter.DEFAULT). -
setConverter
Sets the converter that copies Twitter objects into records (defaults toITwitterConverter.DEFAULT). -
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classProxyReader
-