Class AbstractTwitterReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.internal.lang.IntegrationReader
com.northconcepts.datapipeline.twitter2.AbstractTwitterReader
- Direct Known Subclasses:
TwitterFilterStreamReader,TwitterFollowerReader,TwitterFollowingReader,TwitterSearchReader,TwitterTimelineReader
Base class for Twitter API v2 readers; each page from
search() yields one record per data element.-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final TwitterV2Clientprotected static final intprotected final TwitterCredentialsFields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.intIndicates the maximum number of records that will be obtained by the endpoint.protected StringReturns the pagination token for the next request: empty before the first page and null once no pages remain.protected RecordreadImpl()Overridden by subclasses to read the next record from thisDataReader.protected booleanBuffers the records of the nextsearch()page; returns false when there is no further page to read.protected abstract Responsesearch()Requests the next page of results (seegetNextToken()); returning null ends reading.protected DataExceptionsetException(DataException exception) Stores an exception for the next read to throw and returns it.setMaxResults(int maxResults) Indicates the maximum number of records that will be obtained by the endpoint.Methods inherited from class com.northconcepts.datapipeline.internal.lang.IntegrationReader
openMethods inherited from class com.northconcepts.datapipeline.core.DataReader
addLineage, available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
close, decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
DEFAULT_RETRY_AFTER_SECONDS
protected static final int DEFAULT_RETRY_AFTER_SECONDS- See Also:
-
client
-
twitterCredentials
-
-
Constructor Details
-
AbstractTwitterReader
-
-
Method Details
-
search
protected abstract Response search()Requests the next page of results (seegetNextToken()); returning null ends reading. -
getMaxResults
public int getMaxResults()Indicates the maximum number of records that will be obtained by the endpoint. -
setMaxResults
Indicates the maximum number of records that will be obtained by the endpoint. -
getNextToken
Returns the pagination token for the next request: empty before the first page and null once no pages remain. -
readImpl
Description copied from class:DataReaderOverridden by subclasses to read the next record from thisDataReader. The default implementation ofDataReader.read()now insures that this method will not be called again after it returns anull.If no record is available,
nullwill be returned.Contract for subclasses (see also
docs/authoring/DataReader.md):- Return
nullexactly once to signal end-of-stream. - Do not call this method directly —
DataReader.read()is the template method that wraps exceptions, increments the record count, and tracks lineage. - Do not call
DataEndpoint.incrementRecordCount()here;DataReader.read()already does. - Throw raw exceptions;
DataReader.read()wraps them viaexception(throwable).
- Specified by:
readImplin classDataReader- Throws:
Throwable
- Return
-
readNextBatch
Buffers the records of the nextsearch()page; returns false when there is no further page to read.- Throws:
Throwable
-
setException
Stores an exception for the next read to throw and returns it. -
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataReader
-