Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[OpPerf] PDF Random ops fix#15661

Merged
sandeep-krishnamurthy merged 3 commits intoapache:masterfrom
ChaiBapchya:random_op_fix
Jul 28, 2019
Merged

[OpPerf] PDF Random ops fix#15661
sandeep-krishnamurthy merged 3 commits intoapache:masterfrom
ChaiBapchya:random_op_fix

Conversation

@ChaiBapchya
Copy link
Contributor

@ChaiBapchya ChaiBapchya commented Jul 25, 2019 •
edited
Loading

Description

Fixes #15636
Newly add PDF Random ops - #14617
Caused the opperf.py to error out due to parameter issues.
Hence added new param sample to handle that

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • added param
    Had to add a param Sample (of type NDArray) so as to cater to all the random_pdf_* functions.
    Moreover, I tried playing with different shapes and dimensions. Based of various functions and their definitions, it turns out (2,) is the one-size-fits-all solution. Different functions have different requirement and hence its tough to come up with larger shapes that won't give error on any of these functions.

Comments

Ran python benchmark/opperf/opperf.py successfully without error on CPU & GPU

Error resolved

If you checkout master and run python benchmark/opperf/opperf.py
Will give you following error

Traceback (most recent call last):
  File "benchmark/opperf/opperf.py", line 157, in <module>
    sys.exit(main())
  File "benchmark/opperf/opperf.py", line 142, in main
    final_benchmark_results = run_all_mxnet_operator_benchmarks(ctx=ctx, dtype=dtype)
  File "benchmark/opperf/opperf.py", line 73, in run_all_mxnet_operator_benchmarks
    mxnet_operator_benchmark_results.append(run_mx_random_sampling_operators_benchmarks(ctx=ctx, dtype=dtype))
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/nd_operations/random_sampling_operators.py", line 60, in ru
n_mx_random_sampling_operators_benchmarks
    mx_random_sample_op_results = run_op_benchmarks(mx_random_sample_ops, dtype, ctx, warmup, runs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 127, in run_op_benchmarks
  warmup=warmup, runs=runs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 109, in run_performance_tes
t
    benchmark_result = _run_nd_operator_performance_test(op, inputs, run_backward, warmup, runs, kwargs_list)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 54, in _run_nd_operator_performance_test
    _, _ = benchmark_helper_func(op, warmup, **kwargs_list[0])
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/profiler_utils.py", line 169, in profile_it
    res = func(*args, **kwargs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/ndarray_utils.py", line 79, in nd_forward_and_profile
    res = op(*args, **kwargs)
  File "<string>", line 59, in random_pdf_dirichlet
  File "/home/ubuntu/workspace/incubator-mxnet/python/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/home/ubuntu/workspace/incubator-mxnet/python/mxnet/base.py", line 253, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [22:17:53] src/c_api/../imperative/imperative_utils.h:369: Check failed: num_inputs == infered_num_i
nputs (1 vs. 2) : Operator _random_pdf_dirichlet expects 2 inputs, but got 1 instead.

Stack trace:                                                                                                       [0/1922]
  [bt] (0) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFata
l()+0x32) [0x7fdf8a36f1b2]
  [bt] (1) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(mxnet::imperative::SetNumOutputs(nnvm:
:Op const*, nnvm::NodeAttrs const&, int const&, int*, int*)+0x437) [0x7fdf8d1a4f07]
  [bt] (2) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(MXImperativeInvokeImpl(void*, int, voi
d**, int*, void***, int, char const**, char const**)+0xa50) [0x7fdf8d1a25a0]
  [bt] (3) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(MXImperativeInvokeEx+0x534) [0x7fdf8d1
a4234]
  [bt] (4) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fdfa3a9aec0]
  [bt] (5) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7fdfa3a9a87d]
  [bt] (6) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
 [0x7fdfa3cafe2e]
  [bt] (7) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x12865) [0x7fdfa3cb08
65]
  [bt] (8) python(_PyObject_FastCallDict+0x8b) [0x563b009b2d7b]

@ChaiBapchya ChaiBapchya changed the title [OpPerf] PDF Random ops break the utility [OpPerf] PDF Random ops fix Jul 25, 2019
@abhinavs95
Copy link
Contributor

abhinavs95 commented Jul 26, 2019 •
edited
Loading

@mxnet-label-bot add [Performance, pr-awaiting-review]

@marcoabreu marcoabreu added pr-awaiting-review PR is waiting for code review Performance labels Jul 26, 2019
@ChaiBapchya ChaiBapchya mentioned this pull request Jul 26, 2019
Open
@ChaiBapchya
Copy link
Contributor Author

ChaiBapchya commented Jul 27, 2019

@sandeep-krishnamurthy @apeforest

@sandeep-krishnamurthy sandeep-krishnamurthy merged commit 33ea2db into apache:master Jul 28, 2019
@ChaiBapchya ChaiBapchya deleted the random_op_fix branch July 28, 2019 05:51
anirudhacharya pushed a commit to anirudhacharya/mxnet that referenced this pull request Aug 20, 2019
@ChaiBapchya
[OpPerf] PDF Random ops fix (apache#15661)
ac1577d 
* fix pdf sample for opperf benchmark

* change sample array

* Trigger notification
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@sandeep-krishnamurthy sandeep-krishnamurthy sandeep-krishnamurthy approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Performance pr-awaiting-review PR is waiting for code review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Benchmark/opperf on CPU

4 participants

@ChaiBapchya @abhinavs95 @sandeep-krishnamurthy @marcoabreu