Building Realtime Pipelines in Cloud Data Fusion Reviews
2858 reviews
Niranjan A. · Reviewed over 3 years ago
Oscar R. · Reviewed over 3 years ago
Chris F. · Reviewed over 3 years ago
Thiago d. · Reviewed over 3 years ago
João S. · Reviewed over 3 years ago
Amauri S. · Reviewed over 3 years ago
Randi-Michelle C. · Reviewed over 3 years ago
Akira T. · Reviewed over 3 years ago
Sonit P. · Reviewed over 3 years ago
Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted Change name of subscription in the manual, ex for name "subtest" it works :)
Kacper M. · Reviewed over 3 years ago
Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted
Kacper M. · Reviewed over 3 years ago
ストリーミングでもデータを取り込めるのねってことが分かる・・・？
Yoichi K. · Reviewed over 3 years ago
Adipta M. · Reviewed over 3 years ago
Tran V. · Reviewed over 3 years ago
Alan B. · Reviewed over 3 years ago
I could not add the service account permission.
Aishwarya S. · Reviewed over 3 years ago
Bartlomiej J. · Reviewed over 3 years ago
Nguyen T. · Reviewed over 3 years ago
Nguyen T. · Reviewed over 3 years ago
Nguyen T. · Reviewed over 3 years ago
Mike Z. · Reviewed over 3 years ago
Juan C. · Reviewed over 3 years ago
Basma A. · Reviewed over 3 years ago
Piotr S. · Reviewed over 3 years ago
Kenneth S. · Reviewed over 3 years ago
We do not ensure the published reviews originate from consumers who have purchased or used the products. Reviews are not verified by Google.