See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/668/display/redirect

Changes:

------------------------------------------ [...truncated 1.36 MB...] Monday 30 January 2023 10:07:22 +0000 (0:00:00.081) 1:35:42.347 ******** ok: [localhost] => ut_tests_line: ''

TASK [tensorflow_in_container : (Python 3.10) Extract failed tests] ************ Monday 30 January 2023 10:07:22 +0000 (0:00:00.046) 1:35:42.393 ******** ok: [localhost]

TASK [tensorflow_in_container : (Python 3.10) Show which tests failed] ********* Monday 30 January 2023 10:07:22 +0000 (0:00:00.069) 1:35:42.463 ******** ok: [localhost] => tests_failed: FAILED in

TASK [tensorflow_in_container : (Python 3.10) Convert wheel to manylinux2014 compatible one] *** Monday 30 January 2023 10:07:22 +0000 (0:00:00.072) 1:35:42.535 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.10) Get list of wheel files] ********* Monday 30 January 2023 10:11:51 +0000 (0:04:28.487) 1:40:11.023 ******** ok: [localhost]

TASK [tensorflow_in_container : (Python 3.10) Get filename of latest wheel] **** Monday 30 January 2023 10:11:51 +0000 (0:00:00.367) 1:40:11.391 ******** ok: [localhost]

TASK [tensorflow_in_container : (Python 3.10) move manylinux2014 wheel outside] *** Monday 30 January 2023 10:11:51 +0000 (0:00:00.098) 1:40:11.489 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.10) Remove manylinux2014 wheel] ****** Monday 30 January 2023 10:11:54 +0000 (0:00:02.297) 1:40:13.787 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.10) stop build container] ************ Monday 30 January 2023 10:11:54 +0000 (0:00:00.379) 1:40:14.166 ******** changed: [localhost]

TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Monday 30 January 2023 10:15:33 +0000 (0:03:38.618) 1:43:52.785 ******** changed: [localhost]

TASK [tensorflow_in_container : Remove TensorFlow git sources from previous build] *** Monday 30 January 2023 10:15:33 +0000 (0:00:00.663) 1:43:53.448 ******** skipping: [localhost]

TASK [Fetch TensorFlow] ******************************************************** Monday 30 January 2023 10:15:33 +0000 (0:00:00.037) 1:43:53.486 ******** skipping: [localhost]

TASK [(Python 3.11) patch TensorFlow tree] ************************************* Monday 30 January 2023 10:15:33 +0000 (0:00:00.039) 1:43:53.525 ********

TASK [tensorflow : Create linklibs shell script] ******************************* Monday 30 January 2023 10:15:33 +0000 (0:00:00.133) 1:43:53.659 ******** changed: [localhost]

TASK [tensorflow : Create readme file about unit tests] ************************ Monday 30 January 2023 10:15:34 +0000 (0:00:00.694) 1:43:54.353 ******** changed: [localhost]

TASK [tensorflow : Add to the copied files] ************************************ Monday 30 January 2023 10:15:35 +0000 (0:00:00.699) 1:43:55.052 ******** changed: [localhost]

TASK [tensorflow : Add to the long_description in setup.py] ******************** Monday 30 January 2023 10:15:35 +0000 (0:00:00.367) 1:43:55.420 ******** changed: [localhost]

TASK [tensorflow : Use nightly package requirements] *************************** Monday 30 January 2023 10:15:36 +0000 (0:00:00.362) 1:43:55.782 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) start build container] *********** Monday 30 January 2023 10:15:36 +0000 (0:00:00.390) 1:43:56.173 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) create buildslave bazel cache dir] *** Monday 30 January 2023 10:18:13 +0000 (0:02:36.668) 1:46:32.842 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) create link for bazel cache] ***** Monday 30 January 2023 10:18:13 +0000 (0:00:00.836) 1:46:33.678 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) create link for bazelisk cache] *** Monday 30 January 2023 10:18:14 +0000 (0:00:00.848) 1:46:34.527 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) create link for pip cache] ******* Monday 30 January 2023 10:18:15 +0000 (0:00:00.873) 1:46:35.400 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) copy upper constraints] ********** Monday 30 January 2023 10:18:16 +0000 (0:00:00.886) 1:46:36.287 ******** ok: [localhost]

TASK [tensorflow_in_container : (Python 3.11) install Python packages] ********* Monday 30 January 2023 10:18:17 +0000 (0:00:00.731) 1:46:37.018 ******** changed: [localhost]

TASK [tensorflow_in_container : (Python 3.11) build TensorFlow git] ************ Monday 30 January 2023 10:22:16 +0000 (0:03:59.572) 1:50:36.591 ******** fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Invocation ID: 64c5e6ef-3e1a-421c-842e-a0f284fcbbb4 INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils INFO: Reading rc options for 'build' from /usertools/aarch64.bazelrc: 'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/tf_python --python_path=/usr/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --distinct_host_configuration=false --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3 INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64.bazelrc: --disk_cache=/tf/cache INFO: Found applicable config definition build:ambe in file /usertools/aarch64.bazelrc: --distinct_host_configuration=false --flaky_test_attempts=3 --action_env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin --host_crosstool_top=@ml2014_aarch64_config_aarch64//crosstool:toolchain --crosstool_top=@ml2014_aarch64_config_aarch64//crosstool:toolchain --extra_toolchains=@ml2014_aarch64_config_aarch64//crosstool:toolchain-linux-aarch64 --extra_execution_platforms=@ml2014_aarch64_config_aarch64_platform//:platform --host_platform=@ml2014_aarch64_config_aarch64_platform//:platform --platforms=@ml2014_aarch64_config_aarch64_platform//:platform --repo_env=TF_PYTHON_CONFIG_REPO=@ml2014_aarch64_config_python INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS Loading: Loading: 0 packages loaded Loading: 0 packages loaded Loading: 0 packages loaded Loading: 0 packages loaded Loading: 0 packages loaded Loading: 0 packages loaded currently loading: tensorflow/tools/pip_package Analyzing: target //tensorflow/tools/pip_package:build_pip_package (1 packages loaded, 0 targets configured) Analyzing: target //tensorflow/tools/pip_package:build_pip_package (285 packages loaded, 4232 targets configured) Analyzing: target //tensorflow/tools/pip_package:build_pip_package (584 packages loaded, 33211 targets configured) INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (584 packages loaded, 33522 targets configured). INFO: Found 1 target... [0 / 372] [Prepa] Expanding template tensorflow/lite/python/tflite_convert ... (13 actions, 0 running) ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/com_google_protobuf/BUILD.bazel:848:10: Compiling python/google/protobuf/pyext/descriptor.cc failed: (Exit 1): gcc failed: error executing command /dt10/usr/bin/gcc -MD -MF bazel-out/aarch64-opt/bin/external/com_google_protobuf/_objs/python/google/protobuf/pyext/_message.so/descriptor.pic.d ... (remaining 76 arguments skipped) external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc: In function 'bool google::protobuf::python::_CalledFromGeneratedFile(int)': external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:103:18: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 103 | frame = frame->f_back; | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:109:12: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 109 | if (frame->f_code->co_filename == nullptr) { | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:44, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:114:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 114 | if (PyString_AsStringAndSize(frame->f_code->co_filename, | ^~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:114:7: note: in expansion of macro 'PyString_AsStringAndSize' 114 | if (PyString_AsStringAndSize(frame->f_code->co_filename, | ^~~~~~~~~~~~~~~~~~~~~~~~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:114:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 114 | if (PyString_AsStringAndSize(frame->f_code->co_filename, | ^~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:56:40: note: in definition of macro 'PyString_AsStringAndSize' 56 | PyUnicode_AsUTF8AndSize(ob, (sizep)))) == nullptr \ | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:114:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 114 | if (PyString_AsStringAndSize(frame->f_code->co_filename, | ^~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:59:34: note: in definition of macro 'PyString_AsStringAndSize' 59 | : PyBytes_AsStringAndSize(ob, (charpp), (sizep))) | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:135:12: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 135 | if (frame->f_globals != frame->f_locals) { | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:135:32: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'} 135 | if (frame->f_globals != frame->f_locals) { | ^~ In file included from bazel-out/aarch64-opt/bin/external/local_config_python/python_include/Python.h:42, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.h:37, from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33: bazel-out/aarch64-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'} 22 | typedef struct _frame PyFrameObject; | ^~~~~~ Target //tensorflow/tools/pip_package:build_pip_package failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 62.611s, Critical Path: 4.71s INFO: 13027 processes: 8607 disk cache hit, 4403 internal, 17 local. FAILED: Build did NOT complete successfully FAILED: Build did NOT complete successfully stderr_lines: <omitted> stdout: '' stdout_lines: <omitted>

PLAY RECAP ********************************************************************* localhost : ok=117 changed=89 unreachable=0 failed=1 skipped=13 rescued=0 ignored=3

Monday 30 January 2023 10:23:20 +0000 (0:01:03.523) 1:51:40.115 ******** =============================================================================== tensorflow_in_container : (Python 3.10) run TensorFlow git tests ----- 1392.84s tensorflow_in_container : (Python 3.9) run TensorFlow git tests ------ 1371.77s tensorflow_in_container : (Python 3.8) run TensorFlow git tests ------ 1305.36s tensorflow_in_container : (Python 3.8) Convert wheel to manylinux2014 compatible one - 291.70s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one - 274.90s tensorflow_in_container : (Python 3.10) Convert wheel to manylinux2014 compatible one - 268.49s tensorflow_in_container : (Python 3.11) install Python packages ------- 239.57s tensorflow_in_container : (Python 3.9) stop build container ----------- 219.60s tensorflow_in_container : (Python 3.10) stop build container ---------- 218.62s tensorflow_in_container : (Python 3.8) stop build container ----------- 216.27s tensorflow_in_container : (Python 3.11) start build container --------- 156.67s tensorflow_in_container : (Python 3.8) build TensorFlow git ----------- 107.39s tensorflow_in_container : (Python 3.9) build Python package ------------ 97.08s tensorflow_in_container : (Python 3.10) build Python package ----------- 96.86s tensorflow_in_container : (Python 3.8) build Python package ------------ 95.67s tensorflow_in_container : (Python 3.10) build TensorFlow git ----------- 80.34s tensorflow_in_container : (Python 3.9) build TensorFlow git ------------ 79.01s tensorflow_in_container : (Python 3.11) build TensorFlow git ----------- 63.52s tensorflow : Fetch TensorFlow from git --------------------------------- 21.63s tensorflow_in_container : (Python 3.10) install Python packages -------- 11.48s Build step 'Execute shell' marked build as failure