Details
-
Story
-
Status: Done
-
Normal
-
Resolution: Done
-
None
-
None
-
None
Description
This ticket is to document community requirements and the migration plan to decommission Nexus
Description from: IT-23352
FD.io project data (logs, archived artifacts, & docs) are now being uploaded to Amazon S3 buckets instead of nexus.fd.io. In order to complete the retirement of nexus.fd.io, data currently residing on nexus shall be migrated to amazon S3.
This ticket will be used to document what needs to be migrated from nexus, where it will be migrated to in amazon s3, and track the completion status of the migration.
Attachments
JEditor
Activity
|
Current Status: |
|
38 weeks, 6 days, 15 minutes |
| Transition Name | Total Time in Status | ||||
|---|---|---|---|---|---|
| Create |
|
| To Do |
|
11 seconds |
| Field | Original Value | New Value |
|---|---|---|
| Status | Backlog [ 10701 ] | Selected for Development [ 10714 ] |
| Epic Link |
|
| Component/s | Project: FD.io [ 12502 ] |
| Backlog |
|
5 weeks, 5 days, 22 hours, 5 minutes |
| Status | Selected for Development [ 10714 ] | Backlog [ 10701 ] |
| Epic Link |
|
RELENG-2841 [ 78474 ] |
Andrew Grimberg
made changes -
| Issue Type | Task [ 3 ] | Story [ 10001 ] |
| To Do |
|
31 weeks, 6 days, 6 hours, 16 minutes |
| Status | Backlog [ 10701 ] | Selected for Development [ 10714 ] |
| In Progress |
|
4 seconds |
| Status | Selected for Development [ 10714 ] | In Progress [ 10201 ] |
| Epic Link | RELENG-2841 [ 78474 ] |
Docs - where will we migration docs to
Logs/Artifact archives - S3
Document a list of projects still deploying artifacts to Nexus
| Blocked |
|
6 days, 25 seconds |
| Status | In Progress [ 10201 ] | Blocked [ 10002 ] |
| Epic Link | RELENG-3238 [ 96437 ] |
| Backlog |
|
2 weeks, 2 days, 23 hours, 41 minutes |
| Status | Blocked [ 10002 ] | Backlog [ 10701 ] |
| To Do |
|
3 seconds |
| Status | Backlog [ 10701 ] | Selected for Development [ 10714 ] |
| Backlog |
|
4 days, 19 hours, 47 minutes |
| Status | Selected for Development [ 10714 ] | Backlog [ 10701 ] |
Vratko Polak
added a comment - Comments regarding CSIT: https://lists.fd.io/g/csit-dev/message/4251 
Andrew Yourtchenko
added a comment - VPP docs jobs also deploy to nexus, don’t they ? Vratko Polak
added a comment - Yes.
https://docs.fd.io/vpp/ 
Maciek Konstantynowicz
added a comment - See further comments regarding CSIT project: https://lists.fd.io/g/csit-dev/message/4252.
Maciek Konstantynowicz
added a comment - See further comments regarding CSIT project: https://lists.fd.io/g/csit-dev/message/4252 . | Epic Link | RELENG-3238 [ 96437 ] |
| Fix Version/s | Q4 2020 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 15501 ] |
| Epic Link | RELENG-3238 [ 96437 ] |
| Epic Link | RELENG-3238 [ 96437 ] |
|
| Fix Version/s | Q4 2020 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 15501 ] |
| Epic Link |
|
| Summary | FD.io - Nexus to PackageCloud migration | FD.io - Nexus decommission plan |
| To Do |
|
38 weeks, 1 day, 21 hours, 54 minutes |
| Status | Backlog [ 10701 ] | Selected for Development [ 10714 ] |
Document migration plan
- docs migration
- artifacts migration
- jjb changes
| Epic Link |
|
|
| Fix Version/s | Q3 2021 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 18005 ] |
| Fix Version/s | Q4 2021 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 18413 ] | |
| Fix Version/s | Q3 2021 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 18005 ] |
| Epic Link |
|
|
| Description |
Need to reach out to any projects still using Nexus to see if they are willing to move from Nexus to PackageCloud TSC meeting notes http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2020/fdio-meeting.2020-02-20-16.00.log.html |
| Description | This ticket is to document community requirements and the migration plan to decommission Nexus |
| Link | This issue relates to IT-23352 [ IT-23352 ] |
| Description | This ticket is to document community requirements and the migration plan to decommission Nexus |
This ticket is to document community requirements and the migration plan to decommission Nexus Description from: IT-23352 FD.io project data (logs, archived artifacts, & docs) are now being uploaded to Amazon S3 buckets instead of nexus.fd.io. In order to complete the retirement of nexus.fd.io, data currently residing on nexus shall be migrated to amazon S3. This ticket will be used to document what needs to be migrated from nexus, where it will be migrated to in amazon s3, and track the completion status of the migration. |
| Watchers | Andrew Yourtchenko, Maciek Konstantynowicz, Vanessa Valderrama, Vratko Polak [ Andrew Yourtchenko, Maciek Konstantynowicz, Vanessa Valderrama, Vratko Polak ] | Andrew Yourtchenko, Dave Wallace, Ed Warnicke, Maciek Konstantynowicz, Peter Mikus, Vanessa Valderrama, Vratko Polak [ Andrew Yourtchenko, Dave Wallace, Ed Warnicke, Maciek Konstantynowicz, Peter Mikus, Vanessa Valderrama, Vratko Polak ] |
Peter Mikus
added a comment - From CSIT point of view:
 |
rls1701/ | 2017-02-01 16:02 | - | |
|
rls1704/ | 2017-04-27 19:00 | - | |
|
rls1707/ | 2017-07-21 13:51 | - | |
|
rls1710/ | 2017-10-19 12:53 | - | |
|
rls1801/ | 2018-03-29 08:56 | - | |
|
rls1801_2/ | 2018-06-01 05:53 | - | |
|
rls1804/ | 2018-05-03 08:24 | - | |
|
rls1807/ | 2018-08-08 10:50 | - | |
|
rls1810/ | 2018-11-07 15:15 | - | |
|
rls1901/ | 2019-02-12 15:27 | - | |
|
rls1901_1/ | 2019-03-19 17:35 | - | |
|
rls1901_2/ | 2019-05-27 12:29 | - | |
|
rls1901_3/ | 2019-08-07 11:58 | - | |
|
rls1904/ | 2019-05-10 07:51 | - | |
|
rls1904_1/ | 2019-06-12 17:42 | - | |
|
rls1904_2/ | 2019-08-09 06:25 | - | |
|
rls1908/ | 2019-09-04 05:24 | - | |
|
rls1908_1/ | 2019-09-25 11:19 | - | |
|
rls1908_2/ | 2020-05-06 15:00 | - | |
|
rls2001/ | 2020-03-29 04:22 | - | |
|
rls2005/ | 2020-06-17 09:08 | - | |
|
rls2009/ | 2020-10-14 11:38 | - | |
|
rls2009_lts/ | 2020-12-02 07:17 | - | |
|
rls2101/ | 2021-02-10 08:57 | - | |
|
rls2101_1/ | 2021-08-18 11:35 | - | |
|
rls2106/ | 2021-07-14 15:34 | - |
Please DO NOT copy master folder/branch.
Thank you.
Peter Mikus
added a comment - From CSIT point of view:
https://docs.fd.io/csit/
rls1701/
2017-02-01 16:02
-
rls1704/
2017-04-27 19:00
-
rls1707/
2017-07-21 13:51
-
rls1710/
2017-10-19 12:53
-
rls1801/
2018-03-29 08:56
-
rls1801_2/
2018-06-01 05:53
-
rls1804/
2018-05-03 08:24
-
rls1807/
2018-08-08 10:50
-
rls1810/
2018-11-07 15:15
-
rls1901/
2019-02-12 15:27
-
rls1901_1/
2019-03-19 17:35
-
rls1901_2/
2019-05-27 12:29
-
rls1901_3/
2019-08-07 11:58
-
rls1904/
2019-05-10 07:51
-
rls1904_1/
2019-06-12 17:42
-
rls1904_2/
2019-08-09 06:25
-
rls1908/
2019-09-04 05:24
-
rls1908_1/
2019-09-25 11:19
-
rls1908_2/
2020-05-06 15:00
-
rls2001/
2020-03-29 04:22
-
rls2005/
2020-06-17 09:08
-
rls2009/
2020-10-14 11:38
-
rls2009_lts/
2020-12-02 07:17
-
rls2101/
2021-02-10 08:57
-
rls2101_1/
2021-08-18 11:35
-
rls2106/
2021-07-14 15:34
-
Please DO NOT copy master folder/branch.
Thank you. Andrew Grimberg
made changes -
| Epic Link |
|
|
| Fix Version/s | Q1 2022 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 19600 ] | |
| Fix Version/s | Q4 2021 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 18413 ] |
| Rank | Ranked higher |
Replication of CSIT data .... [done] (both report/doc)
e.g: https://s3-docs.fd.io/csit/rls1701/report/
Replication of VPP to be confirmed (based on requirement)
Peter Mikus
added a comment - - edited Replication of CSIT data .... [done] (both report/doc)
e.g: https://s3-docs.fd.io/csit/rls1701/report/
Replication of VPP to be confirmed (based on requirement) 
Ed Warnicke
added a comment - @Peter, it looks like you are setting up for s3-docs.fd.io to become docs.fd.io once we are comfortable that we are prepared so that we get a smooth transition, is that correct?
Ed Warnicke
added a comment - @Peter, it looks like you are setting up for s3-docs.fd.io to become docs.fd.io once we are comfortable that we are prepared so that we get a smooth transition, is that correct? Peter Mikus
added a comment - @Ed, I would say, we did transition, in all our scripts and ci-man, backlinks we are using S3. The only missing part was old data (21.06 and earlier). To whatever degree any project wants to transfer them.
from CSIT point of view, all done. (140G data)
from VPP point of view, script for migration of "/docs" is now running. (it is very slow as the old data between16.09..21.06 releases counts 1,368,068 files // 32G)
from <any_other_project> point ov view... ????? (e.g back up Honeycomb makes no sense as it is closed). So I suggest to ask via TSC "which projects want to have docs.fd.io transferred? #poll"
Earlier we agreed that transferring jenkins logs is worthless, we are only talking about docs.
Overall I will let you know once VPP is done.
Peter Mikus
added a comment - @Ed, I would say, we did transition, in all our scripts and ci-man, backlinks we are using S3. The only missing part was old data (21.06 and earlier). To whatever degree any project wants to transfer them.
from CSIT point of view, all done. (140G data)
from VPP point of view, script for migration of "/docs" is now running. (it is very slow as the old data between16.09..21.06 releases counts 1,368,068 files // 32G)
from <any_other_project> point ov view... ????? (e.g back up Honeycomb makes no sense as it is closed). So I suggest to ask via TSC "which projects want to have docs.fd.io transferred? #poll"
Earlier we agreed that transferring jenkins logs is worthless, we are only talking about docs.
Overall I will let you know once VPP is done. Ed Warnicke
added a comment - What's the plan to make sure that a doc.fd.io link continues to work after we decommission nexus? Will we simply make doc.fd.io a CNAME for s3-docs.fd.io ?
What is the plan for indexes (the equivalent of this link https://docs.fd.io/vpp/ ) to make sure folks can find stuff?
Ed Warnicke
added a comment - What's the plan to make sure that a doc.fd.io link continues to work after we decommission nexus? Will we simply make doc.fd.io a CNAME for s3-docs.fd.io ?
What is the plan for indexes (the equivalent of this link https://docs.fd.io/vpp/ ) to make sure folks can find stuff? | In Progress |
|
22 weeks, 4 days, 4 hours, 46 minutes |
| Status | Selected for Development [ 10714 ] | In Progress [ 10201 ] |
hagbard
Are you asking about an index doc? Please look over this Amazon documentation and let me know if this is what you're referring to
https://docs.aws.amazon.com/AmazonS3/latest/userguide/IndexDocumentSupport.html
Ed Warnicke
added a comment - That works to a point, except that it appears we would then need to manually create any relevant index document. That might work, but we will need to take some care to get it right, including needing to update those index documents as new versions of things (or new elements of directory structure) land.
Ed Warnicke
added a comment - That works to a point, except that it appears we would then need to manually create any relevant index document. That might work, but we will need to take some care to get it right, including needing to update those index documents as new versions of things (or new elements of directory structure) land.
Peter Mikus
added a comment - Yes to manually create (in fact it already exists) I just need to upload it.
NO!!! to change indexing settings for docc bucket as it would break those docs sites.
Peter Mikus
added a comment - Yes to manually create (in fact it already exists) I just need to upload it.
NO!!! to change indexing settings for docc bucket as it would break those docs sites.
Peter Mikus
added a comment - CNAME for docs.fd.io is fine. but please do not remove s3-docs.fd.io as it is now hardcoded in many places.
Peter Mikus
added a comment - CNAME for docs.fd.io is fine. but please do not remove s3-docs.fd.io as it is now hardcoded in many places. Ed Warnicke
added a comment - Peter: Why is it hard coded? Peter Mikus
added a comment - @Ed: How would you today address s3 only storage? I am not a fan of waiting till someone flip a switch. We are using s3-docs for over a 6mnths. Nexus should be down already. So...
Peter Mikus
added a comment - @Ed: How would you today address s3 only storage? I am not a fan of waiting till someone flip a switch. We are using s3-docs for over a 6mnths. Nexus should be down already. So... Ed Warnicke
added a comment - @Peter: Let me ask a different question that probably will help the discussion:
Where are we referring to s3-docs.fd.io ?
Ed Warnicke
added a comment - @Peter: Let me ask a different question that probably will help the discussion:
Where are we referring to s3-docs.fd.io ? Peter Mikus
added a comment - https://github.com/FDio/site/search?q=s3-docs
https://github.com/FDio/csit/search?q=s3-docs
https://github.com/FDio/ci-management/search?q=s3-docs
Peter Mikus
added a comment - https://github.com/FDio/site/search?q=s3-docs
https://github.com/FDio/csit/search?q=s3-docs
https://github.com/FDio/ci-management/search?q=s3-docs
Discussed during TSC today that we will pick up the decom discussion after the release
| Blocked |
|
2 weeks, 22 hours, 26 minutes |
| Status | In Progress [ 10201 ] | Blocked [ 10002 ] |
pmikus hagbard dwallacelf are we ready to start discussing the decom again?
I think the next steps would be to figure out what to do with the index page and finalize what needs to migrate.
| Epic Link |
|
|
| Fix Version/s | Q2 2022 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 20415 ] | |
| Fix Version/s | Q1 2022 RelEng Operations (Enhancements, Tech Debt, Build, Release Support) [ 19600 ] |
Per discussion with Dave, all artifacts have been migrated. Nexus is ready for decom.
| Done |
|
9 weeks, 5 days, 21 hours, 42 minutes |
| Resolution | Done [ 10000 ] | |
| Status | Blocked [ 10002 ] | Done [ 10200 ] |
Added to the TSC agenda to start discussion