Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename components directory to resources under .dapr directory #1149

Draft
wants to merge 38 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
bd3657f
set dapr run defaults precedence
pravinpushkar Dec 9, 2022
4e6d8b5
dapr init create resources dir and moves existing components content …
pravinpushkar Dec 9, 2022
8358a21
modify tests
pravinpushkar Dec 9, 2022
c3f41e9
fix test
pravinpushkar Dec 9, 2022
6affa75
fix tests
pravinpushkar Dec 12, 2022
1b421b0
add tests
pravinpushkar Dec 12, 2022
0363365
trigger pr checks
pravinpushkar Dec 12, 2022
d9caa15
Update pkg/standalone/common.go
pravinpushkar Dec 13, 2022
2007ed5
Apply suggestions from code review
pravinpushkar Dec 13, 2022
bf0e7ff
review comments
pravinpushkar Dec 13, 2022
d8422a7
review comments
pravinpushkar Dec 13, 2022
e73ec7d
fix error handle on path not present
pravinpushkar Dec 13, 2022
093239b
fix tests
pravinpushkar Dec 13, 2022
5df0795
trigger pr checks
pravinpushkar Dec 13, 2022
125ebc8
readme changes and some more messages today
pravinpushkar Dec 14, 2022
3b55cd5
trigger pr checks
pravinpushkar Dec 14, 2022
3c8020b
Update README.md
pravinpushkar Dec 14, 2022
ff6ce5d
update readme
pravinpushkar Dec 14, 2022
5334d15
Typo
shubham1172 Dec 14, 2022
57cfa8a
Type (2)
shubham1172 Dec 14, 2022
15a894c
Merge branch 'master' into feat/resources_dir
mukundansundar Dec 15, 2022
a4c5f81
few more refactoring
pravinpushkar Dec 15, 2022
03d190a
Merge branch 'master' into feat/resources_dir
mukundansundar Dec 19, 2022
5a624af
fix review comment and failing merge
pravinpushkar Dec 20, 2022
3989aad
make symlink
pravinpushkar Dec 24, 2022
0749c5c
Merge branch 'master' into feat/resources_dir
pravinpushkar Dec 26, 2022
4a421d5
fix tests
pravinpushkar Dec 26, 2022
f73af11
review comments
pravinpushkar Dec 29, 2022
e7ccf04
fix tests
pravinpushkar Dec 29, 2022
771b256
some more refactor
pravinpushkar Dec 30, 2022
c5646d4
merge master and fix conflicts
pravinpushkar Jan 3, 2023
968b5ca
fix tests
pravinpushkar Jan 3, 2023
ed1169c
change few uninstall to uninstallAll and add cleanup in upgrade test
pravinpushkar Jan 4, 2023
5871ad4
Merge branch 'master' into feat/resources_dir
mukundansundar Jan 12, 2023
771d423
Merge branch 'master' into feat/resources_dir
shubham1172 Jan 16, 2023
e471c28
fix conflicts & merge master
pravinpushkar Feb 28, 2023
e2794b0
Merge branch 'master' into feat/resources_dir
pravinpushkar Feb 28, 2023
d605587
lint fix
pravinpushkar Feb 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Output should look like so:

```
⌛ Making the jump to hyperspace...
✅ Downloaded binaries and completed components set up.
✅ Downloaded binaries and completed resources set up.
ℹ️ daprd binary has been installed to $HOME/.dapr/bin.
ℹ️ dapr_placement container is running.
ℹ️ dapr_redis container is running.
Expand All @@ -99,10 +99,12 @@ Output should look like so:

This step creates the following defaults:

1. components folder which is later used during `dapr run` unless the `--resources-path` (`--components-path` is deprecated and will be removed in future releases) option is provided. For Linux/MacOS, the default components folder path is `$HOME/.dapr/components` and for Windows it is `%USERPROFILE%\.dapr\components`.
2. component files in the components folder called `pubsub.yaml` and `statestore.yaml`.
1. resources folder which is later used during `dapr run` unless the `--resources-path` (`--components-path` is deprecated and will be removed in future releases) option is provided. For Linux/MacOS, the default resources folder path is `$HOME/.dapr/resources` and for Windows it is `%USERPROFILE%\.dapr\resources`.
2. resoources files in the resources folder called `pubsub.yaml` and `statestore.yaml`.
3. default config file `$HOME/.dapr/config.yaml` for Linux/MacOS or for Windows at `%USERPROFILE%\.dapr\config.yaml` to enable tracing on `dapr init` call. Can be overridden with the `--config` flag on `dapr run`.

> Note: If there is a `components` directory present in the default dapr installation path `$HOME/.dapr/resources` (for Linux/MacOS) or `%USERPROFILE%\.dapr\resources` (for windows) then `dapr init` will copy the `resources` from the `components` to `resources` directory. This is to ensure safe migration from using `components` to `resources` directory.

#### Slim Init

Alternatively to the above, to have the CLI not install any default configuration files or run Docker containers, use the `--slim` flag with the init command. Only Dapr binaries will be installed.
Expand All @@ -115,13 +117,15 @@ Output should look like so:

```bash
⌛ Making the jump to hyperspace...
✅ Downloaded binaries and completed components set up.
✅ Downloaded binaries and completed resources set up.
ℹ️ daprd binary has been installed to $HOME/.dapr/bin.
ℹ️ placement binary has been installed.
✅ Success! Dapr is up and running. To get started, go here: https://aka.ms/dapr-getting-started
```

>Note: When initializing Dapr with the `--slim` flag only the Dapr runtime binary and the placement service binary are installed. An empty default components folder is created with no default configuration files. During `dapr run` user should use `--resources-path` (`--components-path` is deprecated and will be removed in future releases) to point to a components directory with custom configurations files or alternatively place these files in the default directory. For Linux/MacOS, the default components directory path is `$HOME/.dapr/components` and for Windows it is `%USERPROFILE%\.dapr\components`.
>Note: When initializing Dapr with the `--slim` flag only the Dapr runtime binary and the placement service binary are installed. An empty default `resources` folder is created with no default configuration files. During `dapr run` user should use `--resources-path` (`--components-path` is deprecated and will be removed in future releases) to point to a `resources` directory with custom configurations files or alternatively place these files in the default directory. For Linux/MacOS, the default `resources` directory path is `$HOME/.dapr/resources` and for Windows it is `%USERPROFILE%\.dapr\resources`.

> Note: If there is a `components` directory present in the default dapr installation path `$HOME/.dapr/resources` (for Linux/MacOS) or `%USERPROFILE%\.dapr\resources` (for windows) then `dapr init` will copy the `resources` from the `components` to `resources` directory. This is to ensure safe migration from using `components` to `resources` directory.

#### Install a specific runtime version

Expand Down Expand Up @@ -444,7 +448,7 @@ Launch Dapr and your app:
dapr run --app-id nodeapp --app-port 3000 node app.js
```

Note: To choose a non-default components folder, use the --resources-path(--components-path is deprecated) option.
Note: To choose a non-default resources folder, use the --resources-path option (--components-path is deprecated, see https://github.com/dapr/cli/issues/953 for more information).

Invoke your app:

Expand Down Expand Up @@ -562,8 +566,8 @@ To use a custom path for component definitions

```bash
dapr run --resources-path [custom path]
> Note: --components-path flag is deprecated. It will continue to work until it is removed completely.
```
> Note: --components-path flag is deprecated. It will continue to work until it is removed completely. See https://github.com/dapr/cli/issues/953 for more information.


### List Configurations
Expand Down
2 changes: 1 addition & 1 deletion cmd/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,7 @@ func init() {
RunCmd.Flags().StringVarP(&logLevel, "log-level", "", "info", "The log verbosity. Valid values are: debug, info, warn, error, fatal, or panic")
RunCmd.Flags().IntVarP(&maxConcurrency, "app-max-concurrency", "", -1, "The concurrency level of the application, otherwise is unlimited")
RunCmd.Flags().StringVarP(&protocol, "app-protocol", "P", "http", "The protocol (gRPC or HTTP) Dapr uses to talk to the application")
RunCmd.Flags().StringVarP(&componentsPath, "components-path", "d", standalone.DefaultComponentsDirPath(), "The path for components directory")
RunCmd.Flags().StringVarP(&componentsPath, "components-path", "d", standalone.GetResourcesDir(), "The path for resources directory")
RunCmd.Flags().StringVarP(&resourcesPath, "resources-path", "", "", "The path for resources directory")
pravinpushkar marked this conversation as resolved.
Show resolved Hide resolved
// TODO: Remove below line once the flag is removed in the future releases.
// By marking this as deprecated, the flag will be hidden from the help menu, but will continue to work. It will show a warning message when used.
Expand Down
81 changes: 76 additions & 5 deletions pkg/standalone/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,19 @@ limitations under the License.
package standalone

import (
"fmt"
"os"
path_filepath "path/filepath"
"runtime"

"github.com/dapr/cli/pkg/print"
"github.com/dapr/cli/utils"
)

const (
defaultDaprDirName = ".dapr"
defaultDaprBinDirName = "bin"
defaultComponentsDirName = "components"
defaultConfigFileName = "config.yaml"
defaultDaprDirName = ".dapr"
defaultDaprBinDirName = "bin"
defaultConfigFileName = "config.yaml"
)

func defaultDaprDirPath() string {
Expand All @@ -44,9 +47,77 @@ func binaryFilePath(binaryDir string, binaryFilePrefix string) string {
}

func DefaultComponentsDirPath() string {
return path_filepath.Join(defaultDaprDirPath(), defaultComponentsDirName)
return path_filepath.Join(defaultDaprDirPath(), utils.DefaultComponentsDirName)
}

func DefaultResourcesDirPath() string {
return path_filepath.Join(defaultDaprDirPath(), utils.DefaultResourcesDirName)
}

// GetResourcesDir returns the path to the resources directory if it exists, otherwise it returns the path of components directory.
// TODO: Remove this function and replace all its usage with above defined `DefaultResourcesDirPath` when `--components-path` flag is removed.
func GetResourcesDir() string {
defaultResourcesDirPath := DefaultResourcesDirPath()
if _, err := os.Stat(defaultResourcesDirPath); os.IsNotExist(err) {
return DefaultComponentsDirPath()
}
return defaultResourcesDirPath
}

func DefaultConfigFilePath() string {
return path_filepath.Join(defaultDaprDirPath(), defaultConfigFileName)
}

// moveDir moves files from src to dest. If there are files in src, it deletes the existing files in dest before copying from src and then deletes the src directory.
func moveDir(src, dest string) error {
destFiles, err := os.ReadDir(dest)
if err != nil {
return fmt.Errorf("error reading files from %s: %w", dest, err)
}
srcFiles, err := os.ReadDir(src)
if err != nil {
return fmt.Errorf("error reading files from %s: %w", src, err)
}
if len(srcFiles) > 0 {
// delete the existing files in dest before copying from src if there are files in src.
for _, file := range destFiles {
err = os.Remove(path_filepath.Join(dest, file.Name()))
if err != nil {
return fmt.Errorf("error removing file %s: %w", file.Name(), err)
}
}
print.InfoStatusEvent(os.Stdout, "Moving files from %q to %q", src, dest)
var content []byte
for _, file := range srcFiles {
content, err = os.ReadFile(path_filepath.Join(src, file.Name()))
if err != nil {
return fmt.Errorf("error reading file %s: %w", file.Name(), err)
}
// #nosec G306
err = os.WriteFile(path_filepath.Join(dest, file.Name()), content, 0o644)
if err != nil {
return fmt.Errorf("error writing file %s: %w", file.Name(), err)
}
}
}
err = os.RemoveAll(src)
if err != nil {
return fmt.Errorf("error removing directory %s: %w", src, err)
}
return nil
}

// createSymLink creates a symlink from dirName to symLink.
func createSymLink(dirName, symLink string) error {
if _, err := os.Stat(dirName); err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("directory %s does not exist", dirName)
}
return fmt.Errorf("error reading directory %s: %w", dirName, err)
}
err := os.Symlink(dirName, symLink)
if err != nil {
return fmt.Errorf("error creating symlink from %s to %s: %w", dirName, symLink, err)
}
return nil
}
134 changes: 134 additions & 0 deletions pkg/standalone/common_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
/*
Copyright 2022 The Dapr Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package standalone

import (
"os"
path_filepath "path/filepath"
"testing"

"github.com/stretchr/testify/assert"
)

func TestCreateSymLink(t *testing.T) {
pravinpushkar marked this conversation as resolved.
Show resolved Hide resolved
// create a temp dir to hold the symlink and actual directory.
tempDir := createTempDir(t, "dapr-test", "")
defer cleanupTempDir(t, tempDir)
originalDir := createTempDir(t, "original_dir", tempDir)
existingSymLinkDir := createTempDir(t, "new_name_exist", tempDir)
tests := []struct {
name string
actualDirName string
symLinkName string
expectedError bool
}{
{
name: "create symlink for resources directory",
actualDirName: originalDir,
symLinkName: path_filepath.Join(tempDir, "new_name"),
expectedError: false,
},
{
name: "create symlink when resources directory does not exist",
actualDirName: "invalid-dir",
symLinkName: "new_name",
expectedError: true,
},
{
name: "create symlink when symlink named directory already exists",
actualDirName: originalDir,
symLinkName: existingSymLinkDir,
expectedError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := createSymLink(tt.actualDirName, tt.symLinkName)
assert.Equal(t, tt.expectedError, err != nil)
})
}
}

func TestMoveDir(t *testing.T) {
// create a temp dir to hold the source and destination directory.
tempDir := createTempDir(t, "dapr-test", "")
defer cleanupTempDir(t, tempDir)
// create a file in the source and destination directory.
src1 := createTempDir(t, "src1", tempDir)
dest2 := createTempDir(t, "dest2", tempDir)
srcFile := createTempFile(t, src1, "pubsub.yaml")
destFile := createTempFile(t, dest2, "pubsub-dest.yaml")
tests := []struct {
name string
srcDirName string
destDirName string
expectedError bool
presentFileName string
}{
{
name: "move directory when source directory contains files",
srcDirName: src1,
destDirName: createTempDir(t, "dest1", tempDir),
expectedError: false,
presentFileName: path_filepath.Base(srcFile),
},
{
name: "move directory when source directory does not contain files",
srcDirName: createTempDir(t, "src2", tempDir),
destDirName: dest2,
expectedError: false,
presentFileName: path_filepath.Base(destFile),
},
{
name: "move directory when source directory does not exists",
srcDirName: path_filepath.Join(tempDir, "non-existent-source-dir"),
destDirName: createTempDir(t, "dest3", tempDir),
expectedError: true,
},
{
name: "move directory when destination directory does not exists",
srcDirName: createTempDir(t, "src4", tempDir),
destDirName: path_filepath.Join(tempDir, "non-existent-dir-dir"),
expectedError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := moveDir(tt.srcDirName, tt.destDirName)
assert.Equal(t, tt.expectedError, err != nil)
if tt.presentFileName != "" {
// check if the files are moved correctly.
assert.FileExists(t, path_filepath.Join(tt.destDirName, tt.presentFileName))
}
})
}
}

func createTempDir(t *testing.T, tempDirName, containerDir string) string {
dirName, err := os.MkdirTemp(containerDir, tempDirName)
assert.NoError(t, err)
return dirName
}

func createTempFile(t *testing.T, tempDirName, fileName string) string {
file, err := os.CreateTemp(tempDirName, fileName)
assert.NoError(t, err)
defer file.Close()
return file.Name()
}

func cleanupTempDir(t *testing.T, dirName string) {
err := os.RemoveAll(dirName)
assert.NoError(t, err)
}
10 changes: 5 additions & 5 deletions pkg/standalone/run_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,17 @@ func assertArgumentNotEqual(t *testing.T, key string, expectedValue string, args
}

func setupRun(t *testing.T) {
componentsDir := DefaultComponentsDirPath()
resourcesDir := DefaultResourcesDirPath()
configFile := DefaultConfigFilePath()
err := os.MkdirAll(componentsDir, 0o700)
err := os.MkdirAll(resourcesDir, 0o700)
assert.Equal(t, nil, err, "Unable to setup components dir before running test")
file, err := os.Create(configFile)
file.Close()
assert.Equal(t, nil, err, "Unable to create config file before running test")
}

func tearDownRun(t *testing.T) {
err := os.RemoveAll(DefaultComponentsDirPath())
err := os.RemoveAll(DefaultResourcesDirPath())
assert.Equal(t, nil, err, "Unable to delete default components dir after running test")
err = os.Remove(DefaultConfigFilePath())
assert.Equal(t, nil, err, "Unable to delete default config file after running test")
Expand All @@ -87,7 +87,7 @@ func assertCommonArgs(t *testing.T, basicConfig *RunConfig, output *RunOutput) {
assertArgumentEqual(t, "app-max-concurrency", "-1", output.DaprCMD.Args)
assertArgumentEqual(t, "app-protocol", "http", output.DaprCMD.Args)
assertArgumentEqual(t, "app-port", "3000", output.DaprCMD.Args)
assertArgumentEqual(t, "components-path", DefaultComponentsDirPath(), output.DaprCMD.Args)
assertArgumentEqual(t, "components-path", GetResourcesDir(), output.DaprCMD.Args)
assertArgumentEqual(t, "app-ssl", "", output.DaprCMD.Args)
assertArgumentEqual(t, "metrics-port", "9001", output.DaprCMD.Args)
assertArgumentEqual(t, "dapr-http-max-request-size", "-1", output.DaprCMD.Args)
Expand Down Expand Up @@ -148,7 +148,7 @@ func TestRun(t *testing.T) {
EnableProfiling: false,
ProfilePort: 9090,
Protocol: "http",
ComponentsPath: DefaultComponentsDirPath(),
ComponentsPath: GetResourcesDir(),
AppSSL: true,
MetricsPort: 9001,
MaxRequestBodySize: -1,
Expand Down
Loading