Skip to content

New website: review from [2/3] to [3/3]: New Pipelines#6

Open
ThomasDelteil wants to merge 101 commits intonew_website_delete_contentfrom
new_website_pipeline_2
Open

New website: review from [2/3] to [3/3]: New Pipelines#6
ThomasDelteil wants to merge 101 commits intonew_website_delete_contentfrom
new_website_pipeline_2

Conversation

@ThomasDelteil
Copy link
Owner

Review the new pipeline in isolation here

ThomasDelteil and others added 2 commits August 13, 2019 17:10
Co-authored-by: Aaron Markham <markhama@amazon.com>
Co-authored-by: Thomas Delteil <thomas.delteil1@gmail.com>
Co-authored-by: Thomas Delteil <thomas.delteil1@gmail.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>
@vishaalkapoor
Copy link

I'm familiar with the md changes and have reviewed the Jenkins pipeline code. LGTM.



build_docs_setup() {
build_folder="docs/_build"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no set -ex for this step ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point


popd

GZIP=-9 tar zcvf $artifacts_path $docs_build_path

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this line do ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this zips up with high compression (GZIP=-9) the generated artifact in a specific path so that it can then be picked up by CI for building the entire website as the sum of its components


build_version_doc() {

# $1 is the list of branch or tag to build

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no set -ex here ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function can be deleted. It's only used for the current website.

@@ -0,0 +1,218 @@
# Creating custom operators with numpy

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ptrendx pointed out that AMP docs are missing

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's correct, some tutorials are missing and will be part of a post-launch re-alignment

Copy link
Collaborator

@aaronmarkham aaronmarkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggested changes and comments.
I'm ignoring the /docs/python_docs and other folders since I don't think they're intended to be in this PR.

node(NODE_LINUX_CPU) {
ws('workspace/docs') {
timeout(time: max_time, unit: 'MINUTES') {
archiveArtifacts 'docs/_build/full_website.tgz'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be replaced with upload to S3 once we have a working CI-dev env for it.

// utils.pack_lib('full_website', 'docs/_build/full_website.tgz', false)

// archive so the publish pipeline can access the artifact
archiveArtifacts 'docs/_build/full_website.tgz'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be switched to S3 when a test environment is made available on CI-dev.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

utils.docker_run('ubuntu_cpu_jekyll', 'build_docs_small', false)

// Publish preview to S3
sh "cd docs/_build/html && ci/other/ci_deploy_doc.sh ${env.BRANCH_NAME} ${env.BUILD_NUMBER}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not useful until there's a website serving process attached to S3.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will comment out until you / someone setups an alternative setup

Co-Authored-By: Aaron Markham <markhama@amazon.com>
ThomasDelteil and others added 30 commits September 17, 2019 11:49
* Set monitor callback basic support

* Trigger CI

* Add base.pyi and ndarray.pyx

* Change not supported to experimental and check for both static_shape and static_alloc
* add aaron to codeowners

* retrigger CI

* ci
* add shard API

* more balanced

* fix a bug

* Update test_gluon_data.py
Add test

Fix

Fix sanity

Fix build failure

Use array_function protocol only when np.version >= 1.17

Add unit tests for array ufunc protocol

Fix pylint

Reformat

Fix build

Fix

Refactor test suite for numpy interoperability
* Fix README Build Status

* Add labels to Jenkins Badges

* Add more line breaks
* improve dataloader signals and messages

* address comments

* fix spawn tests on windows
* normal implemented

* numpy behavior normal imlemented

* retrigger CI

* retrigger CI

* regrigger ci

* add normal parameter check

* add raise for normal

* remove dead code
* Fix mkldnn subgraph with float64

* Fix ci

Change-Id: I0bb4e8d7a0a534aa661601887cc633cb9c4fcadf

* Fix test

Change-Id: I96c529abe7adb6def90a22f03b3432263ef12fda

* Update dmlc-core

Change-Id: I472fb7bbffc16ed8c36494ab49838b08c59b2f12

* pin to offical dmlc

Change-Id: I5a27dc83b892bf8fcb34bb089449d1d3b6e9beed

* Fix GPU CI

Change-Id: I285947e01bdb0651c2c7830ed4eb76931a09b754

* Fix GPU CI

Change-Id: I6f23b51d6bda44f6ae18766ebe390118740bb9c7
* Adding new website files

Co-authored-by: Thomas Delteil <thomas.delteil1@gmail.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>

* removing old content

Co-authored-by: Thomas Delteil <thomas.delteil1@gmail.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>

* add license header

* fixing links and url + css

* fixing tutorials

* exclude python_docs folder when building prod site

This allows things to coexist until a following PR deletes this file.

* license header fixes

* exclude static site from license header check

* exclude static site and sphinx theme

* exclude static_site and cpp_docs from sphinx

* add licenses and excludes

* fix headers

* restore file for legacy builds

* empty commit to trigger CI

* update submodule to correct one

* delete website check

* add placeholder website check

* add placeholder website check, for reals this time

* trigger CI

* trigger PR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.