> 1. Incremental loading ('Create table' is not ticked')
Skyvia replication package has the "Incremental Updates" check box, which allows processing only the records, changed or added in a cloud source since last package run. For more information, please refer to https://skyvia.com/resources/docs/index.html?replication_package.htm
If you select the "Create Tables" and "Drop Tables" check boxes together with "Incremental Updates", the tables are created/dropped only on the first run of the package. They are not (re)created during next runs.
> 2. Add or remove fields after the first load
Skyvia caches metadata of available objects for cloud sources. So, if a field is added, removed or changed in a NetSuite object or you add or delete an object, the cache should be refreshed. To do this, you need to open your NetSuite connection and click 'clear' for the Metadata Cache parameter. For more information, please refer to https://skyvia.com/resources/docs/index.html?netsuite_connections.htm
After this you need to edit your package, then edit your task and click Refresh. You also need to re-create tables in Azure Data Warehouse, so that they don't have these columns. If you use incremental updates, then clear the LastSyncDate parameter and select "Create Tables" and "Drop Tables". You package will delete table from Azure Data Warehouse, create a new table with actual NetSuite schema, and copy all records from NetSuite to Azure Data Warehouse.
If you do not clear metadata cache or refresh replication package, SQL queries that access non-existing (deleted) fields may be generated, and this will cause an error. It doesn't matter if you edit the Azure Data Warehouse schema manually. For any NetSuite schema changes, you need to clear the metadata cache, edit the replication package and re-create tables in Azure Data Warehouse.
Whenever you modify the package schema, the LastSyncDate parameter is reset, and the next replication is performed as in the first time.
You need to note that Skyvia support for NetSuite is somewhat limited. Skyvia cannot extract data from NetSuite array fields, and thus, cannot replicate these data.
For example, Skyvia cannot extract items from NetSuite transactions - CashRefund, CashSale, Check, CreditMemo, Estimate, Invoice, Opportunity, PurchaseOrder, ReturnAuthorization, SalesOrder, etc. There also are other tables, not transactions, which also are not integrated completely. When you replicate such tables, certain data from them is not replicated.
Data from other tables, like Customer, Contact, Campaign, etc. can be replicated completely.