Skip to content

Commit bb60dd4

Browse files
authored
Merge pull request #4280 from mathesar-foundation/0.2.1
Release v0.2.1
2 parents 9fd9a00 + 5cfd4d0 commit bb60dd4

File tree

143 files changed

+3336
-2529
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

143 files changed

+3336
-2529
lines changed

.env.example

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
ALLOWED_HOSTS=.localhost,127.0.0.1,[::1]
2-
SECRET_KEY=2gr6ud88x=(p855_5nbj_+7^bw-iz&n7ldqv%94mjaecl+b9=4
1+
ALLOWED_HOSTS=".localhost, 127.0.0.1, [::1]"
2+
SECRET_KEY="2gr6ud88x=(p855_5nbj_+7^bw-iz&n7ldqv%94mjaecl+b9=4"

.github/workflows/test-and-lint-code.yml

+2
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,7 @@ jobs:
165165
matrix:
166166
py-version: [3.9-bookworm, 3.10-bookworm, 3.11-bookworm, 3.12-bookworm, 3.13-bookworm]
167167
pg-version: [13, 14, 15, 16, 17]
168+
init-db-args: ["", "--encoding=SQL_ASCII"]
168169
steps:
169170
- uses: actions/checkout@v4
170171
- name: Copy env file
@@ -178,6 +179,7 @@ jobs:
178179
env:
179180
PYTHON_VERSION: ${{ matrix.py-version }}
180181
PG_VERSION: ${{ matrix.pg-version }}
182+
POSTGRES_INITDB_ARGS: ${{ matrix.init-db-args }}
181183

182184
python_lint:
183185
name: Run Python linter

DEVELOPER_GUIDE.md

+6
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,12 @@ We use [pytest](https://docs.pytest.org) for our backend tests.
105105
docker exec mathesar_dev_db /bin/bash sql/run_tests.sh
106106
```
107107
108+
### Running scenario tests
109+
110+
```
111+
./run_api_tests.sh
112+
```
113+
108114
## Front end development
109115
110116
- All the front end code is in the `mathesar_ui` directory.

Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ ENTRYPOINT ["./dev-run.sh"]
9898

9999
#=========== STAGE: PRODUCTION ===============================================#
100100

101-
from base as production
101+
FROM base AS production
102102

103103
# Install prod requirements
104104
RUN pip install --no-cache-dir -r requirements.txt

README.md

+72-60
Original file line numberDiff line numberDiff line change
@@ -1,110 +1,122 @@
11
<p align="center">
22
<img src="https://user-images.githubusercontent.com/845767/218793207-a84a8c9e-d147-40a8-839b-f2b5d8b1ccba.png" width=450px alt="Mathesar logo"/>
33
</p>
4-
<p align="center"><b>An intuitive UI for managing data, for users of all technical skill levels. Built on Postgres.</b></p>
4+
<p align="center"><b>Intuitive spreadsheet-like interface that lets users of all technical skill levels view, edit, query, and collaborate on Postgres data directly—self hosted, with native Postgres access control.</b></p>
55
<p align="center">
66
<img alt="License" src="https://img.shields.io/github/license/mathesar-foundation/mathesar">
77
<img alt="GitHub closed issues" src="https://img.shields.io/github/issues-closed/mathesar-foundation/mathesar">
88
<img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/w/mathesar-foundation/mathesar">
99
</p>
1010

1111
<p align="center">
12-
<a href="https://mathesar.org?ref=github-readme" target="_blank">Website</a> • <a href="https://docs.mathesar.org?ref=github-readme" target="_blank">Docs</a> • <a href="https://demo.mathesar.org?ref=github-readme" target="_blank">Live Demo</a> • <a href="https://wiki.mathesar.org/en/community/matrix" target="_blank">Matrix (chat)</a> • <a href="https://discord.gg/enaKqGn5xx" target="_blank">Discord</a> • <a href="https://wiki.mathesar.org/" target="_blank">Wiki</a>
12+
<a href="https://mathesar.org?ref=github-readme" target="_blank">Website</a> • <a href="https://docs.mathesar.org?ref=github-readme-top" target="_blank">Docs</a> • <a href="https://wiki.mathesar.org/en/community/matrix" target="_blank">Matrix (chat)</a> • <a href="https://discord.gg/enaKqGn5xx" target="_blank">Discord</a> • <a href="https://wiki.mathesar.org/" target="_blank">Contributor Wiki</a>
1313
</p>
1414

1515

1616
# Mathesar
17+
Mathesar is a web application that makes working with PostgreSQL databases both simple and powerful. It empowers users of all technical skill levels to view, edit, query, and collaborate on data with a familiar spreadsheet-like interface—no code needed. It’s self hosted, can be deployed in minutes, and works directly with PostgreSQL databases, schemas, and tables without extra abstractions. The project is 100% open source and maintained by Mathesar Foundation, a 501(c)(3) nonprofit.
1718

18-
Mathesar is a straightforward open source tool that provides a **spreadsheet-like interface** to a PostgreSQL **database**. Our web-based interface helps you and your collaborators work with data more independently and comfortably – **no technical skills needed**.
19-
20-
You can use Mathesar to build **data models**, **enter data**, and even **build reports**. You host your own Mathesar installation, which gives you ownership, privacy, and control of your data.
19+
Mathesar is as scalable as Postgres and supports any size or complexity of data, making it ideal for workflows involving production databases. It requires minimal setup, and integrates into your existing infrastructure. Because Mathesar is self-hosted, your data never leaves your servers, and access control based on Postgres roles and privileges keeps your database secure without adding unnecessary risk.
2120

2221
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
2322
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
2423
**Table of Contents**
2524

26-
- [Sponsors](#sponsors)
2725
- [Status](#status)
28-
- [Join our community!](#join-our-community)
29-
- [Screenshots](#screenshots)
30-
- [Live Demo](#live-demo)
26+
- [Install Mathesar](#install-mathesar)
27+
- [Join our community](#join-our-community)
28+
- [Contribute to Mathesar](#contribute-to-mathesar)
3129
- [Features](#features)
32-
- [Self-hosting](#self-hosting)
30+
- [Screeenshots](#screeenshots)
31+
- [Connecting a database](#connecting-a-database)
32+
- [Adding collaborators](#adding-collaborators)
33+
- [Viewing a Postgres schema](#viewing-a-postgres-schema)
34+
- [Working with tables](#working-with-tables)
35+
- [Finding a nested record](#finding-a-nested-record)
36+
- [Linking two tables together](#linking-two-tables-together)
37+
- [Managing table permissions](#managing-table-permissions)
38+
- [Viewing a single record with related records](#viewing-a-single-record-with-related-records)
39+
- [Viewing an Exploration (query result)](#viewing-an-exploration-query-result)
40+
- [Building an Exploration (query)](#building-an-exploration-query)
41+
- [Disconnecting a database](#disconnecting-a-database)
42+
- [Mathesar in action](#mathesar-in-action)
3343
- [Our motivation](#our-motivation)
34-
- [Contributing](#contributing)
3544
- [Bugs and troubleshooting](#bugs-and-troubleshooting)
3645
- [License](#license)
3746

3847
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
3948

40-
## Sponsors
41-
Our top sponsors! Become a sponsor on [GitHub](https://github.com/sponsors/mathesar-foundation) or [Open Collective](https://opencollective.com/mathesar).
42-
43-
<table>
44-
<tbody>
45-
<tr>
46-
<td align="center" valign="top" width="14.28%">
47-
<a href="https://www.thingylabs.io/">
48-
<img src="https://user-images.githubusercontent.com/287034/226116547-cd28e16a-4c89-4a01-bc98-5a19b02ab1b2.png" width="100px;" alt="Thingylabs GmbH"/>
49-
<br />
50-
<sub><b>Thingylabs GmbH</b></sub>
51-
</a>
52-
<br />
53-
</td>
54-
</tr>
55-
</tbody>
56-
</table>
57-
5849
## Status
5950
- [x] **Public Alpha**: You can install and deploy Mathesar on your server. Go easy on us!
60-
- [ ] **Public Beta**: Stable and feature-rich enough to implement in production
61-
- [ ] **Public**: Production-ready
62-
63-
We are currently in the **public alpha** stage.
64-
65-
## Join our community!
66-
The Mathesar team is on [Matrix](https://wiki.mathesar.org/en/community/matrix) (chat service). We also have [mailing lists](https://wiki.mathesar.org/en/community/mailing-lists) and the core team discusses day-to-day work on our developer mailing list.
67-
68-
## Screenshots
69-
70-
![crm-table](https://user-images.githubusercontent.com/287034/220773466-1790a826-923e-47a8-8f7e-1edb67970a16.png)
51+
- [x] **Public Beta**: Stable and feature-rich enough to implement in production
52+
- [ ] **Public**: Widely used in production environments
7153

72-
![authors-filter](https://user-images.githubusercontent.com/287034/220773378-78e05984-5f0f-4ed2-9682-b75ca0f6867c.png)
54+
We are currently in the **public beta** stage.
7355

74-
![talks-with-topics](https://user-images.githubusercontent.com/287034/220773633-0a4ff810-a1e1-476f-b5b0-2667ba97f07a.png)
56+
## Install Mathesar
57+
Please see [our documentation](https://docs.mathesar.org/?ref=github-readme-installing) for instructions on installing Mathesar on your own server.
7558

76-
![author-record](https://user-images.githubusercontent.com/287034/220773738-a3fd0dda-cf16-45ed-a8ef-4e40647bb074.png)
59+
## Join our community
60+
The Mathesar team is on [Matrix](https://wiki.mathesar.org/en/community/matrix) (chat service). We also have [mailing lists](https://wiki.mathesar.org/en/community/mailing-lists) and the core team discusses day-to-day work on our developer mailing list.
7761

78-
![arxiv-schema](https://user-images.githubusercontent.com/287034/220773323-bd6ffb31-835b-4df5-981e-dae6341d42bb.png)
79-
80-
![db-page](https://user-images.githubusercontent.com/287034/220773522-8c1c1483-2389-4f5e-83b2-e54836983035.png)
81-
82-
## Live Demo
83-
Check out a [live demo of Mathesar here](https://demo.mathesar.org)!
62+
### Contribute to Mathesar
63+
We actively encourage contribution! Get started by reading our [Contributor Guide](./CONTRIBUTING.md).
8464

8565
## Features
8666
- **Built on Postgres**: Connect to an existing Postgres database or set one up from scratch.
67+
- **Install in minutes**: Install using Docker in minutes, integrate into any existing infrastructure.
68+
- **Postgres-based access control**: Use existing Postgres roles within Mathesar's UI, or set up your own.
69+
- **Interoperable with other tools**: Mathesar works harmoniously alongside your database and thousands of other tools in the Postgres ecosystem.
8770
- **Set up your data models**: Easily create and update Postgres schemas and tables.
8871
- **Data entry**: Use our spreadsheet-like interface to view, create, update, and delete table records.
8972
- **Filter, sort, and group**: Quickly slice your data in different ways.
9073
- **Query builder**: Use our Data Explorer to build queries without knowing anything about SQL or joins.
74+
- **Import and export data**: Import and export data into Mathesar easily to work with your data elsewhere.
9175
- **Schema migrations**: Transfer columns between tables in two clicks.
92-
- **Uses Postgres features**: Mathesar uses and manipulates Postgres schemas, primary keys, foreign keys, constraints and data types. e.g. "Links" in the UI are foreign keys in the database.
93-
- **Custom data types**: Custom data types for emails and URLs (more coming soon), validated at the database level.
94-
- **Basic access control**: Users can have Viewer (read-only), Editor (can only edit data, but not data structure), or Manager (can edit both data and its structure) roles.
76+
- **Uses Postgres features**: Mathesar uses and manipulates Postgres schemas, primary keys, foreign keys, constraints and data types. e.g. "Relationships" in the UI are foreign keys in the database.
77+
- **Custom data types**: Custom data types for emails and URLs, validated at the database level.
9578

96-
## Self-hosting
97-
Please see [our documentation](https://docs.mathesar.org/) for instructions on installing Mathesar on your own server.
79+
## Screeenshots
80+
### Connecting a database
81+
![connect-db](https://github.com/user-attachments/assets/c115ad21-e501-4992-bf84-54758a321f41)
9882

99-
## Our motivation
100-
Mathesar is a non-profit project. Our goal is to make understanding and working with data easy for everyone.
83+
### Adding collaborators
84+
![add-collaborator](https://github.com/user-attachments/assets/90c65f3f-3edb-4bf3-b00c-586019c78765)
10185

102-
Databases have been around for a long time and solve common data problems really well. But working with databases often requires custom software. Or complex tooling that people struggle to get their heads around.
86+
### Viewing a Postgres schema
87+
![schema-page](https://github.com/user-attachments/assets/305d0d79-fba8-4044-954d-bc511c935321)
10388

104-
We want to make existing database functionality more accessible, for users of all technical skill levels.
89+
### Working with tables
90+
![table-inspector](https://github.com/user-attachments/assets/bb3bbcd7-ef12-4304-9d5c-8220b188d2f5)
10591

106-
## Contributing
107-
We actively encourage contribution! Get started by reading our [Contributor Guide](./CONTRIBUTING.md).
92+
### Finding a nested record
93+
![record-selector](https://github.com/user-attachments/assets/b3d12cb4-e90b-458b-8e19-8371c1332557)
94+
95+
### Linking two tables together
96+
![relationship-creation](https://github.com/user-attachments/assets/faf8e82c-d165-495a-a9eb-bf76ec9a9283)
97+
98+
### Managing table permissions
99+
![table-permissions](https://github.com/user-attachments/assets/094d3c98-8c4c-4cfa-aad5-6fd3faa30473)
100+
101+
### Viewing a single record with related records
102+
![record-page](https://github.com/user-attachments/assets/00774552-2acb-4a01-87d6-1813f579e757)
103+
104+
### Viewing an Exploration (query result)
105+
![viewing-exploration](https://github.com/user-attachments/assets/0df680d3-621d-466a-9915-d560001a3931)
106+
107+
### Building an Exploration (query)
108+
![building-exploration](https://github.com/user-attachments/assets/7ff100fe-cd26-4229-af3b-a5a5bb70b0b7)
109+
110+
### Disconnecting a database
111+
![disconnect-db](https://github.com/user-attachments/assets/48905983-a632-4632-b2c1-b0f25c3b6e75)
112+
113+
## Mathesar in action
114+
https://github.com/user-attachments/assets/6bdfb178-17b4-4abf-aac4-9781e1d841ab
115+
116+
## Our motivation
117+
Using databases shouldn't require technical expertise or expensive, closed-off tools. Databases are incredibly powerful, but they're often trapped behind complex interfaces that are hard to use or limit how people can access and share their data. We want to change that by building user-friendly tools that unlock the power of existing databases without sacrificing accessibility, portability, or extensibility.
118+
119+
Mathesar is our answer: an open-source platform designed to unlock the full potential of PostgreSQL, one of the most powerful and trusted open-source databases. Mathesar is easy to use, interoperable, and extensible, while also giving you complete control over your data. As a nonprofit, we're committed to keeping Mathesar 100% open source and available to everyone—because better ways to work with data mean better decisions, and better decisions lead to a better world.
108120

109121
## Bugs and troubleshooting
110122
If you run into problems, refer to our [troubleshooting guide](./TROUBLESHOOTING.md).

api_tests/test_happy_db_setups.py

+18-3
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ def test_create_mathesar_db_internal(admin_rpc_call):
8282
result = admin_rpc_call(
8383
'databases.setup.create_new',
8484
database='mathesar',
85-
sample_data=['library_management']
85+
sample_data=['library_management', 'bike_shop']
8686
)
8787
assert set(result.keys()) == set(['configured_role', 'database', 'server'])
8888
internal_db = result['database']
@@ -267,8 +267,8 @@ def test_schemas_list(admin_rpc_call):
267267
'schemas.list',
268268
database_id=1
269269
)
270-
# Should have `public` and `Library Management`
271-
assert len(result) == 2
270+
# Should have `public`, `Bike Shop`, and `Library Management`
271+
assert len(result) == 3
272272
library_management_oid = [
273273
s['oid'] for s in result if s['name'] == 'Library Management'
274274
][0]
@@ -370,6 +370,21 @@ def test_intern_can_now_access_books_table(intern_rpc_call):
370370
assert len(records['results']) == 20
371371

372372

373+
def test_schema_delete(admin_rpc_call):
374+
admin_rpc_call(
375+
'schemas.delete',
376+
schema_oids=[library_management_oid],
377+
database_id=internal_db_id,
378+
)
379+
result = admin_rpc_call(
380+
'schemas.list',
381+
database_id=1
382+
)
383+
# Should have `public` and `Bike Shop` remaining`
384+
assert len(result) == 2
385+
assert library_management_oid not in [s['oid'] for s in result]
386+
387+
373388
def test_disconnect_fails_when_dependencies(admin_session):
374389
response = admin_session.post(
375390
RPC_ENDPOINT,

config/settings/common_settings.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ def pipe_delim(pipe_string):
147147
SECRET_KEY = os.environ.get('SECRET_KEY', default="2gr6ud88x=(p855_5nbj_+7^gw-iz&n7ldqv%94mjaecl+b9=4")
148148

149149
# SECURITY WARNING: don't run with debug turned on in production!
150-
DEBUG = bool(os.environ.get('DEBUG', default=False))
150+
DEBUG = os.environ.get('DEBUG') in ['t', 'true', 'True']
151151

152152
ALLOWED_HOSTS = [i.strip() for i in os.environ.get('ALLOWED_HOSTS', default=".localhost, 127.0.0.1, [::1]").split(',')]
153153

config/settings/production.py

-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
from config.settings.common_settings import * # noqa
22

33
# Override default settings
4-
DEBUG = False
54
MATHESAR_MODE = 'PRODUCTION'
65
MATHESAR_ANALYTICS_URL = 'https://analytics.mathesar.dev/collect-analytics-reports'
76
MATHESAR_INIT_REPORT_URL = 'https://analytics.mathesar.dev/collect-initial-report'

conftest.py

+4-31
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,8 @@ def _create_schema(schema_name, engine, schema_mustnt_exist=True):
172172
if schema_mustnt_exist:
173173
assert schema_name not in created_schemas
174174
logger.debug(f'creating {schema_name}')
175-
_create_schema_if_not_exists_via_sql_alchemy(schema_name, engine)
175+
with engine.begin() as conn:
176+
conn.execute(text(f'CREATE SCHEMA IF NOT EXISTS {schema_name}'))
176177
schema_oid = _get_schema_oid_from_name(schema_name, engine)
177178
db_name = engine.url.database
178179
created_schemas_in_this_engine = created_schemas.setdefault(db_name, {})
@@ -187,38 +188,14 @@ def _create_schema(schema_name, engine, schema_mustnt_exist=True):
187188
# Handle schemas being renamed during test
188189
schema_name = _get_schema_name_from_oid(schema_oid, engine)
189190
if schema_name:
190-
_drop_schema_via_name(engine, schema_name, cascade=True)
191+
with engine.begin() as conn:
192+
conn.execute(text(f'DROP SCHEMA IF EXISTS {schema_name} CASCADE'))
191193
logger.debug(f'dropping {schema_name}')
192194
except OperationalError as e:
193195
logger.debug(f'ignoring operational error: {e}')
194196
logger.debug('exit')
195197

196198

197-
def _create_schema_if_not_exists_via_sql_alchemy(schema_name, engine):
198-
return _execute_msar_func_with_engine(
199-
engine, 'create_schema_if_not_exists', schema_name
200-
).fetchone()[0]
201-
202-
203-
def _execute_msar_func_with_engine(engine, func_name, *args):
204-
"""
205-
Execute an msar function using an SQLAlchemy engine.
206-
207-
This is temporary scaffolding.
208-
209-
Args:
210-
engine: an SQLAlchemy engine for connecting to a DB
211-
func_name: The unqualified msar function name (danger; not sanitized)
212-
*args: The list of parameters to pass
213-
"""
214-
conn_str = str(engine.url)
215-
with psycopg.connect(conn_str) as conn:
216-
return conn.execute(
217-
f"SELECT msar.{func_name}({','.join(['%s'] * len(args))})",
218-
args
219-
)
220-
221-
222199
def _get_schema_name_from_oid(oid, engine, metadata=None):
223200
schema_info = _reflect_schema(engine, oid=oid, metadata=metadata)
224201
if schema_info:
@@ -249,10 +226,6 @@ def _reflect_schema(engine, name=None, oid=None, metadata=None):
249226
return schema_info
250227

251228

252-
def _drop_schema_via_name(engine, name, cascade=False):
253-
_execute_msar_func_with_engine(engine, 'drop_schema', name, cascade).fetchone()
254-
255-
256229
# Seems to be roughly equivalent to mathesar/database/base.py::create_mathesar_engine
257230
# TODO consider fixing this seeming duplication
258231
# either way, both depend on Django configuration. can that be resolved?

db/schemas.py

+2-12
Original file line numberDiff line numberDiff line change
@@ -45,15 +45,5 @@ def create_schema(schema_name, conn, owner_oid, description=None):
4545
return db_conn.exec_msar_func(conn, 'create_schema', schema_name, owner_oid, description).fetchone()[0]
4646

4747

48-
def drop_schema_via_oid(conn, id, cascade=False):
49-
"""
50-
Drop a schema by its OID.
51-
52-
If no schema exists with the given oid, an exception will be raised.
53-
54-
Args:
55-
conn: a psycopg connection
56-
id: the OID of the schema to drop.
57-
cascade: Whether to drop the dependent objects.
58-
"""
59-
db_conn.exec_msar_func(conn, 'drop_schema', id, cascade).fetchone()
48+
def drop_schemas(conn, sch_oids):
49+
db_conn.exec_msar_func(conn, 'drop_schemas', sch_oids)

0 commit comments

Comments
 (0)