mv the folder "iLiT-Sample-for-Tensorflow" following new structure. (#308)

* create

* rm wrong file

* push missed files

* add ci prepare cmd

* add sudo in env

* fix the env by clone to private env

* fix env setting

* mv the ilit to new folder, clear the output of ipy

* rm temp files

* Lqnguyen branch3 (#210)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <[email protected]>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <[email protected]>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <[email protected]>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <[email protected]>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <[email protected]>

* Update json file.

* Sync with master.

* Update bitonic-sort code sample according to the latest guideline.

Signed-off-by: Loc Nguyen <[email protected]>

Co-authored-by: JoeOster <[email protected]>

* Lqnguyen branch1 (#201)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <[email protected]>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <[email protected]>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <[email protected]>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <[email protected]>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <[email protected]>

* Update json file.

* Sync with master.

* Update the PrefixSum code sample according to the latest guidelines.

Signed-off-by: Loc Nguyen <[email protected]>

* Update based on comments from reviewer.

Signed-off-by: Loc Nguyen <[email protected]>

* Restructure the Usage function.

Signed-off-by: Loc Nguyen <[email protected]>

Co-authored-by: JoeOster <[email protected]>

* Lqnguyen branch2 (#209)

* Add bitonic-sort sample.

* Add a note about common file in README.

Signed-off-by: Loc Nguyen <[email protected]>

* Move 1d_HeatTransfer sample to open source GitHub.

Signed-off-by: Loc Nguyen <[email protected]>

* Updating License file to remove date

* Adding Buffer Object approach.

* Add comment about the location of dpc_common.hpp.

* New sample: Prefix Sum.

* Remove new sample.

* New code sample PrefixSum in ParallelPatterns.

Signed-off-by: Loc Nguyen <[email protected]>

* Integrate MPI code sample with dpc_reduce code sample.

* Update README.md

* Update main.cpp

* Integrate MPI with latest dpc_reduce for beta09.

* Update README.md

* Update main.cpp

* Update main.cpp

* Update README.md

* Update CXX to icpx and compiler option for beta09.

Signed-off-by: Loc Nguyen <[email protected]>

* Add "export I_MPI_CXX=dpcpp" in sample.json file.

Signed-off-by: Loc Nguyen <[email protected]>

* Update json file.

* Sync with master.

* Update 1d_HeatTransfer code sample according to the new guideline.

* Add comment about dpc_common.hpp .

Signed-off-by: Loc Nguyen <[email protected]>

Co-authored-by: JoeOster <[email protected]>

* namespace change for montecarlo (#208)

* Adding mandelbrot sample to the repository

Signed-off-by: vmadanan <[email protected]>

* Adding changes to mandelbrot to remove libsycl-complex.so dependency

* namespace change for Monte Carlo

* Updated samples to newest coding guidelines

* Updating samples- Mandelbrot, DCT and MonteCarlo with newest coding guidelines

* Adding changes to buffer and accessor declarations (#214)

* Initial commit for iso3dfd_dpcpp code sample

Signed-off-by: Gogar, Sunny L <[email protected]>

* Update License.txt

* Update sample.json

* Adding iso3dfd_omp_offload and changing dpc++ compile for windows to dpcpp

* Delete .nfs000000043228fc3f00000140

* Removing build directory accidently checked in

* Update sample.json

Fixing a missing comma

* Adding couple of changes as per Paul's recommendation

* Updating some variable names as per guidelines

* Moving iso3dfd_omp_offload to C++ folder

* Fixing a windows related error about missing std:: for tranform

* Adding algorithm header explicity in iso3dfd.h

* Fixing the sample.json to eliminate recent errors

* Adding changes to buffer and accessor declarations

* Update samples for beta10 release (#207)

* Update simple add sample

Signed-off-by: Maria, Moushumi <[email protected]>

* Update make files

Signed-off-by: Maria, Moushumi <[email protected]>

* Update fpga make file

Signed-off-by: Maria, Moushumi <[email protected]>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

Co-authored-by: JoeOster <[email protected]>

* folder structures changes following saumya's request (#217)

* Beta10 GZIP performance update (#204)

* Beta10 GZIP update -- use USM for data transfer

Signed-off-by: Audrey Kertesz <[email protected]>

* Trivial change to re-trigger CI

Signed-off-by: Audrey Kertesz <[email protected]>

* Update top level README (#222)

* Update top-level README and improve format

Signed-off-by: Audrey Kertesz <[email protected]>

* Minor formatting update

Signed-off-by: Audrey Kertesz <[email protected]>

* Fix path to oneDPL for Beta10  (#224)

* initial commit of openMP example.

Signed-off-by: todd.erdner <[email protected]>

* Initial commit of the dpc_reduce

Signed-off-by: todd.erdner <[email protected]>

* added guid to sample.json

Signed-off-by: todd.erdner <[email protected]>

* fixed sample.json files.

Signed-off-by: todd.erdner <[email protected]>

* fixed the include files.  Somehow I copied a slightly old repo and it still had <chrono> and the omp_common.hpp file.  They have been removed.

Signed-off-by: todd.erdner <[email protected]>

* added license.txt file ran through formating tool one more time removed all calls to "std::endl" and replaced with " \n"

Signed-off-by: todd.erdner <[email protected]>

* renamed license.txt to License.txt

Signed-off-by: todd.erdner <[email protected]>

* added "ciTests" to the sample.json file.  It passed the check.

Signed-off-by: todd.erdner <[email protected]>

* fixed make error

Signed-off-by: todd.erdner <[email protected]>

* fixed sample.json

Signed-off-by: todd.erdner <[email protected]>

* removed "2020" from the License.txt file due to update guidelines.

Signed-off-by: todd.erdner <[email protected]>

* added comment regarding where you can find dpc_common in both files per Paul's comments.

Signed-off-by: todd.erdner <[email protected]>

* Modified names of the functions to represent what they do (ie. calc_pi_*) per suggestion from Paul.

Signed-off-by: todd.erdner <[email protected]>

* initial check-in to the C++ repo

Signed-off-by: todd.erdner <[email protected]>

* put correct comment on dpc_common.hpp

Signed-off-by: todd.erdner <[email protected]>

* added commenting indicating where they can find corresponding include files.

Signed-off-by: todd.erdner <[email protected]>

* added comment line

Signed-off-by: todd.erdner <[email protected]>

* removed openMP repo from DPC++ as it will be moved to C++ directory

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* fixed category line in sample.json to match exact text expected.

* removing openMP from the DPC directory.  It has been moved to C++ directory.

* fixed tf_init call

Signed-off-by: todd.erdner <[email protected]>

* removed all calls into PSTL internal logic.  This is what was causing fails between beta08 and beta09.

Signed-off-by: todd.erdner <[email protected]>

* fixed env variable to run on CPU

Signed-off-by: todd.erdner <[email protected]>

* update Readme file to include information about setting
env variable to allocate more memory for any runs
on the cpu

Signed-off-by: todd.erdner <[email protected]>

* added option in Cmake file to support unnamed lambda option.   You need this to compile if the environment doesn't have this set by default.

Signed-off-by: todd.erdner <[email protected]>

* path to output file from compile has changed.  it no longer seems to create the src directory.

* started to remove get_access and change it to accessor name()

Signed-off-by: todd.erdner <[email protected]>

* fixed remaining get_access

Signed-off-by: todd.erdner <[email protected]>

* removed commented out old code

Signed-off-by: todd.erdner <[email protected]>

* Fixed path in Cmakelists.txt to suport both beta10 and beta09.  The location of the oneDPL
library changed between the two releases.

* Update CMakeLists.txt

Co-authored-by: JoeOster <[email protected]>

* Added new Sample (TensorFlow Multinode Training with Horovod) (#197)

* Added new Sample (TensorFlow Multinode Training with Horovod)

Signed-off-by: Shailen Sobhee <[email protected]>

* Fixed assert reported by bandit code checker tool.

Signed-off-by: Shailen Sobhee <[email protected]>

* Fix CI issue (MPI bug) - Upload to new folder structure

Signed-off-by: Shailen Sobhee <[email protected]>

* Minor little fix in sample.json; A comma was missing.

Signed-off-by: Shailen Sobhee <[email protected]>

* Removed old references to old folder structure

Signed-off-by: Shailen Sobhee <[email protected]>

* Update third_party_programs.txt (#221)

* Updating License  file to no date in the title /*
 * Copyright (c) 2020 Intel Corporation
 *
 * This program and the accompanying materials are made available under the
 * terms of the The MIT License which is available at
 * https://opensource.org/licenses/MIT.
 *
 * SPDX-License-Identifier: MIT
 */

* Update README.md

* Fix FPGA entries

* Update README.md

Updates per request of sranikonda

* Update README.md

* removing duplicate samples after transfering to dwarves folders

* Update Makefile.win

changing compiler name from "dpcpp-cl" to "dpcpp"

* Update Makefile.win

* Update Makefile.win.fpga

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update README.md

* Update README.md

* Update from Legal Approval of 10/05/2020

Co-authored-by: akertesz <[email protected]>

* Update Buffers/Accessors according to latest coding guidelines (Matrix_multiply Advisor and VTune). (#215)

* TBB Samples Migration

Signed-off-by: root <[email protected]>

* Addressing PR Change Requests

Signed-off-by: root <[email protected]>

* Fill in "Purpose" Section of both README files.

Signed-off-by: root <[email protected]>

* Remove binary and build files

Signed-off-by: root <[email protected]>

* include dpc_common header, remove exception handler, fix json files. (all changes apply to both samples)

Signed-off-by: root <[email protected]>

* include dpc_common headers, remove exception handlers (both samples)

Signed-off-by: root <[email protected]>

* Fix README files, include header files for windows

* Remove namespace, end files, use "std::iota", fix README

Signed-off-by: root <[email protected]>

* fix README

Signed-off-by: root <[email protected]>

* Fix "matrix_multiply" samples failures on Windows.

* buffer/accessor updates for coding guidelines (matrix mul).

Co-authored-by: root <[email protected]>

* oneMKL sample updates for beta10 (#213)

* Jupyter notebooks update as per the latest guidelines (#223)

* updated the simplied version of the accessors, used auto for parallel_for
index

* using vector.size() instead of the global variables as per the comments

* fixed the typo. Also check the output vector size

* Updated Readme to add the include files path for dpc_common.hpp
Updated the cpp file with the comments on dev_utilities folder

* Updated the Jupyter notebooks as per the beta10 guidelines <[email protected]>

* removed sample.json as these are jupyter notebooks <[email protected]>

* removed some checkpoint files that are not necessary <[email protected]>

* removed unwanted files <[email protected]>

* removed unwanted checkpoint files <[email protected]>

* Samples: block APSP and merge SPMV (#219)

* Update simple add sample

Signed-off-by: Maria, Moushumi <[email protected]>

* Update make files

Signed-off-by: Maria, Moushumi <[email protected]>

* Update fpga make file

Signed-off-by: Maria, Moushumi <[email protected]>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

* Samples: block apsp and merge spmv

* Add readme files

* Update readme file

* Update sample.json

Co-authored-by: JoeOster <[email protected]>

* move TF GS sample to new folder structure according to Saumya's direction (#227)

* Update sample.json (#228)

* Update simple add sample

Signed-off-by: Maria, Moushumi <[email protected]>

* Update make files

Signed-off-by: Maria, Moushumi <[email protected]>

* Update fpga make file

Signed-off-by: Maria, Moushumi <[email protected]>

* Add dpc_common.hpp

* Update sample.json

* Fix Makefile.win

* Update Makefile.win

* Update sample.json

* Remove dpc_common.hpp

* Update VS project file

* Update README.md

* Update sample.json

* Add stb

* Update read me file

* Initial commit

* Update License.txt

* Change location of matrix multiplication sample

* Fix matrix mul sample VS project file

* Update samples for beta10 release

* Fix for Windows

* Fix for FPGA

* Fix for FPGA

* Fix for FPGA to support both beta09 and beta10

* Add header comment

* Samples: block apsp and merge spmv

* Add readme files

* Update readme file

* Update sample.json

* Update sample.json

Co-authored-by: JoeOster <[email protected]>

* Edit for readme and some json files (#229)

* Updating License  file to no date in the title /*
 * Copyright (c) 2020 Intel Corporation
 *
 * This program and the accompanying materials are made available under the
 * terms of the The MIT License which is available at
 * https://opensource.org/licenses/MIT.
 *
 * SPDX-License-Identifier: MIT
 */

* Update README.md

* Fix FPGA entries

* Update README.md

Updates per request of sranikonda

* Update README.md

* removing duplicate samples after transfering to dwarves folders

* Update Makefile.win

changing compiler name from "dpcpp-cl" to "dpcpp"

* Update Makefile.win

* Update Makefile.win.fpga

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update README.md

* Update README.md

* Update from Legal Approval of 10/05/2020

* Create README.md

* Add files via upload

* Update README.md

minor modifications to content, purpose and key implementation details.

* Update sample.json

aligned description with readme

* Update README.md

reshuffled parts of the purpose and implementation details and abstracted a few key concepts into better summaries.

* Update sample.json

synched description with readme.

* Update README.md

Co-authored-by: akertesz <[email protected]>
Co-authored-by: tomlenth <[email protected]>

* Changed folder structure (#220)

* Moved model zoo sample to new directory (#216)

* moved model zoo sample to new directory

* added runipy dependency installation

* added error handling

* minor fix

* Updating buffers/accessors for TBB Samples according to coding guidelines. Update CMake files to use defaults. (#230)

* TBB Samples Migration

Signed-off-by: root <[email protected]>

* Addressing PR Change Requests

Signed-off-by: root <[email protected]>

* Fill in "Purpose" Section of both README files.

Signed-off-by: root <[email protected]>

* Remove binary and build files

Signed-off-by: root <[email protected]>

* include dpc_common header, remove exception handler, fix json files. (all changes apply to both samples)

Signed-off-by: root <[email protected]>

* include dpc_common headers, remove exception handlers (both samples)

Signed-off-by: root <[email protected]>

* Fix README files, include header files for windows

* Remove namespace, end files, use "std::iota", fix README

Signed-off-by: root <[email protected]>

* fix README

Signed-off-by: root <[email protected]>

* Fix "matrix_multiply" samples failures on Windows.

* buffer/accessor updates for coding guidelines (matrix mul).

* Update buffers/accessors for TBB Samples. Update CMake files to use defaults.

Co-authored-by: root <[email protected]>

* Update oneVPL samples for Beta10 (#218)

* Add computed_tomography sample (#212)

* create

* rm wrong file

* push missed files

* add ci prepare cmd

* add sudo in env

* fix the env by clone to private env

* fix env setting

* mv the ilit to new folder, clear the output of ipy

* rm temp files

* change structure

* rebase the update

* rm .gitkeep

* update for new API and config for ilit 1.0 in golden release

* update the script to prepare running env

Co-authored-by: Zhang, Jianyu <[email protected]>
Co-authored-by: lqnguyen <[email protected]>
Co-authored-by: JoeOster <[email protected]>
Co-authored-by: vmadananth <[email protected]>
Co-authored-by: slgogar <[email protected]>
Co-authored-by: Moushumi <[email protected]>
Co-authored-by: Louie Tsai <[email protected]>
Co-authored-by: akertesz <[email protected]>
Co-authored-by: terdner <[email protected]>
Co-authored-by: Shailen Sobhee <[email protected]>
Co-authored-by: clevels <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: petercad <[email protected]>
Co-authored-by: praveenkk123 <[email protected]>
Co-authored-by: tomlenth <[email protected]>
Co-authored-by: Jing Xu <[email protected]>
Co-authored-by: Jitendra Patil <[email protected]>
Co-authored-by: Marc Valle <[email protected]>

Author: 19 people

AI-and-Analytics/Getting-Started-Samples/IntelLowPrecisionOptimization_GettingStarted/.gitkeep

@@ -0,0 +1,17 @@
+model:
+  name: hello_world
+  framework: tensorflow                          # possible values are tensorflow, mxnet and pytorch
+  inputs: x
+  outputs: Identity
+
+evaluation:
+  accuracy:
+    metric:
+      topk: 1
+
+tuning:
+  accuracy_criterion:
+    relative: 0.01                              # the tuning target of accuracy loss percentage: 1%
+  exit_policy:
+    timeout: 100                                   # tuning timeout (seconds)
+  random_seed: 100                               # random seed

AI-and-Analytics/Getting-Started-Samples/IntelLowPrecisionOptimization_GettingStarted/iLiT-Sample-for-Tensorflow/alexnet.yaml

@@ -1,50 +1,50 @@
-import ilit
-import alexnet
-import math
-import mnist_dataset
-
-
-def save_int8_frezon_pb(q_model, path):
-    from tensorflow.python.platform import gfile
-    f = gfile.GFile(path, 'wb')
-    f.write(q_model.as_graph_def().SerializeToString())
-    print("Save to {}".format(path))
-
-
-class Dataloader(object):
-    def __init__(self, batch_size):
-        self.batch_size = batch_size
-
-    def __iter__(self):
-        x_train, y_train, label_train, x_test, y_test, label_test = mnist_dataset.read_data()
-        batch_nums = math.ceil(len(x_test) / self.batch_size)
-
-        for i in range(batch_nums - 1):
-            begin = i * self.batch_size
-            end = (i + 1) * self.batch_size
-            yield x_test[begin: end], label_test[begin: end]
-
-        begin = (batch_nums - 1) * self.batch_size
-        yield x_test[begin:], label_test[begin:]
-
-
-def auto_tune(input_graph_path, yaml_config, batch_size):
-    fp32_graph = alexnet.load_pb(input_graph_path)
-    tuner = ilit.Tuner(yaml_config)
-    dataloader = Dataloader(batch_size)
-
-    q_model = tuner.tune(
-        fp32_graph,
-        q_dataloader=dataloader,
-        eval_func=None,
-        eval_dataloader=dataloader)
-    return q_model
-
-
-yaml_file = "alexnet.yaml"
-batch_size = 200
-fp32_frezon_pb_file = "fp32_frezon.pb"
-int8_pb_file = "alexnet_int8_model.pb"
-
-q_model = auto_tune(fp32_frezon_pb_file, yaml_file, batch_size)
+import ilit
+import alexnet
+import math
+import mnist_dataset
+
+
+def save_int8_frezon_pb(q_model, path):
+    from tensorflow.python.platform import gfile
+    f = gfile.GFile(path, 'wb')
+    f.write(q_model.as_graph_def().SerializeToString())
+    print("Save to {}".format(path))
+
+
+class Dataloader(object):
+    def __init__(self, batch_size):
+        self.batch_size = batch_size
+
+    def __iter__(self):
+        x_train, y_train, label_train, x_test, y_test, label_test = mnist_dataset.read_data()
+        batch_nums = math.ceil(len(x_test) / self.batch_size)
+
+        for i in range(batch_nums - 1):
+            begin = i * self.batch_size
+            end = (i + 1) * self.batch_size
+            yield x_test[begin: end], label_test[begin: end]
+
+        begin = (batch_nums - 1) * self.batch_size
+        yield x_test[begin:], label_test[begin:]
+
+
+def auto_tune(input_graph_path, yaml_config, batch_size):
+    fp32_graph = alexnet.load_pb(input_graph_path)
+    quan = ilit.Quantization(yaml_config)
+    dataloader = Dataloader(batch_size)
+
+    q_model = quan(
+        fp32_graph,
+        q_dataloader=dataloader,
+        eval_func=None,
+        eval_dataloader=dataloader)
+    return q_model
+
+
+yaml_file = "alexnet.yaml"
+batch_size = 200
+fp32_frezon_pb_file = "fp32_frezon.pb"
+int8_pb_file = "alexnet_int8_model.pb"
+
+q_model = auto_tune(fp32_frezon_pb_file, yaml_file, batch_size)
 save_int8_frezon_pb(q_model, int8_pb_file)

AI-and-Analytics/Getting-Started-Samples/IntelLowPrecisionOptimization_GettingStarted/iLiT-Sample-for-Tensorflow/License.txt renamed to AI-and-Analytics/Getting-Started-Samples/iLiT-Sample-for-Tensorflow/License.txt

@@ -17,6 +17,34 @@
     "- Compare Quantized Model"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### iLiT Release and Sample \n",
+    "\n",
+    "This sample code is always updated for the iLiT release in latest oneAPI release.\n",
+    "\n",
+    "If you want to get the sample code for old oneAPI release, please checkout the old sample code release by git tag.\n",
+    "\n",
+    "1. Check tags\n",
+    "\n",
+    "```\n",
+    "git pull\n",
+    "git tag\n",
+    "\n",
+    "2021.1-beta08\n",
+    "2021.1-beta09\n",
+    "2021.1-beta10\n",
+    "\n",
+    "```\n",
+    "\n",
+    "2. Checkout old code release\n",
+    "```\n",
+    "git checkout 2021.1-beta10\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -40,6 +68,7 @@
     "import ilit\n",
     "print(ilit.__path__)\n",
     "\n",
+    "\n",
     "import numpy as np"
    ]
   },
@@ -267,11 +296,11 @@
     "\n",
     "def auto_tune(input_graph_path, yaml_config, batch_size):    \n",
     "    fp32_graph = alexnet.load_pb(input_graph_path)\n",
-    "    tuner = ilit.Tuner(yaml_config)\n",
+    "    quan = ilit.Quantization(yaml_config)\n",
     "    assert(tuner)\n",
     "    dataloader = Dataloader(batch_size)\n",
     "    assert(dataloader)\n",
-    "    q_model = tuner.tune(\n",
+    "    q_model = quan(\n",
     "                        fp32_graph,\n",
     "                        q_dataloader=dataloader,\n",
     "                        eval_func=None,\n",