Class ShopifyApiReader
java.lang.Object
com.northconcepts.datapipeline.core.DataObject
com.northconcepts.datapipeline.core.Endpoint
com.northconcepts.datapipeline.core.DataEndpoint
com.northconcepts.datapipeline.core.DataReader
com.northconcepts.datapipeline.internal.lang.IntegrationReader
com.northconcepts.datapipeline.shopify.ShopifyApiReader
- Direct Known Subclasses:
ShopifyCustomerReader,ShopifyEventReader,ShopifyInventoryItemReader,ShopifyInventoryLevelReader,ShopifyLocationReader,ShopifyOrderReader,ShopifyProductReader
Base class for readers that page through a Shopify Admin API resource using
page_info cursors and retry
requests that fail with a 429 (rate limit) error.-
Nested Class Summary
Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
DataEndpoint.State -
Field Summary
FieldsFields inherited from class com.northconcepts.datapipeline.core.DataReader
fieldLineage, recordLineageFields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEYFields inherited from class com.northconcepts.datapipeline.core.Endpoint
BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZEFields inherited from class com.northconcepts.datapipeline.core.DataObject
id, log, name, TIMESTAMP_FORMAT -
Constructor Summary
ConstructorsConstructorDescriptionShopifyApiReader(String domain, String token) Creates a reader for the store atdomain, a base URL such ashttps://example.myshopify.com, using an access token. -
Method Summary
Modifier and TypeMethodDescriptionaddExceptionProperties(DataException exception) Adds this endpoint's current state to aDataException.voidclose()Indicates that this endpoint has finished reading or writing.intReturns the number of records requested per page (defaults to 250, the maximum).longReturns the delay in milliseconds before the first retry of a rate-limited request (defaults to 1000).intReturns the maximum number of attempts, including the first, for a rate-limited request (defaults to 5).protected StringReturns thepage_infocursor of the next page to request or null for the first page.protected abstract ArrayValuegetRecordsFromResponse(Record record) Returns the records in a response frominvokeRequest(), such as itsordersarray; null or empty ends reading.Returns the strategy for the delay before retrying a rate-limited request (defaults toRetryStrategy.EXPONENTIAL_BACKOFF).protected abstract RecordFetches the page for thegetNextPageInfo()cursor (up togetBatchSize()records); must not return null.voidopen()Makes this endpoint ready for reading or writing.protected RecordreadImpl()Overridden by subclasses to read the next record from thisDataReader.setBatchSize(int batchSize) Sets the number of records requested per page; values above 250 are capped and values of 0 or less reset it to 250.setInitialRetryDelay(long initialRetryDelay) Sets the delay in milliseconds before the first retry of a rate-limited request (defaults to 1000).setMaxRetryCount(int maxRetryCount) Sets the maximum number of attempts, including the first, for a rate-limited request (defaults to 5).setRetryStrategy(IRetryStrategy retryStrategy) Sets the strategy for the delay before retrying a rate-limited request (defaults toRetryStrategy.EXPONENTIAL_BACKOFF).Methods inherited from class com.northconcepts.datapipeline.core.DataReader
addLineage, available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skipMethods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toStringMethods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription
-
Field Details
-
service
-
-
Constructor Details
-
ShopifyApiReader
Creates a reader for the store atdomain, a base URL such ashttps://example.myshopify.com, using an access token.
-
-
Method Details
-
getBatchSize
public int getBatchSize()Returns the number of records requested per page (defaults to 250, the maximum). -
setBatchSize
Sets the number of records requested per page; values above 250 are capped and values of 0 or less reset it to 250. -
getRetryStrategy
Returns the strategy for the delay before retrying a rate-limited request (defaults toRetryStrategy.EXPONENTIAL_BACKOFF). -
setRetryStrategy
Sets the strategy for the delay before retrying a rate-limited request (defaults toRetryStrategy.EXPONENTIAL_BACKOFF). -
getMaxRetryCount
public int getMaxRetryCount()Returns the maximum number of attempts, including the first, for a rate-limited request (defaults to 5). -
setMaxRetryCount
Sets the maximum number of attempts, including the first, for a rate-limited request (defaults to 5). -
getInitialRetryDelay
public long getInitialRetryDelay()Returns the delay in milliseconds before the first retry of a rate-limited request (defaults to 1000). -
setInitialRetryDelay
Sets the delay in milliseconds before the first retry of a rate-limited request (defaults to 1000). -
getNextPageInfo
Returns thepage_infocursor of the next page to request or null for the first page. -
open
Description copied from class:DataEndpointMakes this endpoint ready for reading or writing.- Overrides:
openin classIntegrationReader- Throws:
DataException
-
close
Description copied from class:DataEndpointIndicates that this endpoint has finished reading or writing.- Overrides:
closein classDataEndpoint- Throws:
DataException
-
readImpl
Description copied from class:DataReaderOverridden by subclasses to read the next record from thisDataReader. The default implementation ofDataReader.read()now insures that this method will not be called again after it returns anull.If no record is available,
nullwill be returned.Contract for subclasses (see also
docs/authoring/DataReader.md):- Return
nullexactly once to signal end-of-stream. - Do not call this method directly —
DataReader.read()is the template method that wraps exceptions, increments the record count, and tracks lineage. - Do not call
DataEndpoint.incrementRecordCount()here;DataReader.read()already does. - Throw raw exceptions;
DataReader.read()wraps them viaexception(throwable).
- Specified by:
readImplin classDataReader- Throws:
Throwable
- Return
-
invokeRequest
Fetches the page for thegetNextPageInfo()cursor (up togetBatchSize()records); must not return null. Reading continues with another page only if the response has a non-emptynextfield. -
getRecordsFromResponse
Returns the records in a response frominvokeRequest(), such as itsordersarray; null or empty ends reading. -
addExceptionProperties
Description copied from class:EndpointAdds this endpoint's current state to aDataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.- Overrides:
addExceptionPropertiesin classDataReader
-