Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add datasets importing functionality #201

Open
wants to merge 60 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
de78f6b
feat(server/helpers): Add import search
shivam-tripathi Jul 12, 2018
acc103f
feat(helpers/utils): Add function to get import models
shivam-tripathi Jul 18, 2018
e60449e
feat(helpers/utils): Add function to get import model by name
shivam-tripathi Jul 18, 2018
56670ee
feat(helpers/utils): Add JSDocs for import functions
shivam-tripathi Jul 18, 2018
b272cc3
feat(server/helpers/search): Add import support in _fetchEntityModels…
shivam-tripathi Jul 18, 2018
c0b695e
feat(server/helpers/): Update licenses
shivam-tripathi Jul 18, 2018
870e0f0
feat(client/components/pages/search): Update search with message for …
shivam-tripathi Jul 18, 2018
cf4680e
feat(client/components/pages/parts/search-results): Update search res…
shivam-tripathi Jul 18, 2018
203dc05
feat(client/containers/layout): Add panel link to recent imports
shivam-tripathi Jul 12, 2018
90be205
feat(scripts): Update build-client-js and watch-client-js to track re…
shivam-tripathi Jul 31, 2018
0de46fa
feat(server/routes/imports/recent): Add routes to handle recent imports
shivam-tripathi Jul 31, 2018
3ad38bf
feat(client/controllers/recentImports): Add recentImports controller
shivam-tripathi Jul 31, 2018
8de2735
feat(client/components/pages/parts/recentImports): Add part to hold r…
shivam-tripathi Jul 31, 2018
53b0b6f
feat(client/components/pages/recentImports): Add base component to di…
shivam-tripathi Jul 31, 2018
61657de
feat(server/routes): Add imports route
shivam-tripathi Jul 31, 2018
0344296
feat(client/helpers/utils): Add function to extract imports url
shivam-tripathi Jul 31, 2018
4513011
feat(client/helpers/pagination-props): Add function to determine pagi…
shivam-tripathi Jul 31, 2018
08b4d0a
feat(scripts): Add build and watch command in client scripts for impo…
shivam-tripathi Aug 3, 2018
1426f1f
feat(client/helpers/props): Add function to extract import entity props
shivam-tripathi Aug 3, 2018
2854bff
feat(client/controllers/import-entity): Add import-entity controller
shivam-tripathi Aug 3, 2018
913c54b
feat(client/helpers/import-entity): Add import-entity helper functions
shivam-tripathi Aug 3, 2018
208fc7e
feat(client/components/pages/import-entities/footer): Add footer for …
shivam-tripathi Aug 3, 2018
49639b8
feat(src/client/components/pages/import-entities/title): Add title fr…
shivam-tripathi Aug 3, 2018
2f78a55
feat(src/client/components/pages/import-entities/footer): Remove unne…
shivam-tripathi Aug 3, 2018
21915cd
feat(client/components/pages/import-entities): Add import-entities pages
shivam-tripathi Aug 3, 2018
37ae6a2
feat(server/routes/import-entity): Refactor, add more routes
shivam-tripathi Aug 3, 2018
ae662d0
feat(recent-imports): Refactor code
shivam-tripathi Aug 10, 2018
06e8219
feat(scripts/client): Update client build and watch scripts
shivam-tripathi Aug 10, 2018
45fc92d
feat(package.json): Add uuid module
shivam-tripathi Aug 10, 2018
f5fba4b
feat(client/components/pages/entities): Make compulsory props of imag…
shivam-tripathi Aug 10, 2018
fdc8f17
refactor(client/components/pages/parts/recent-import-results)
shivam-tripathi Aug 10, 2018
48bd1b4
feat(client/discard-import-entity): Add confirm discard vote cast page
shivam-tripathi Aug 10, 2018
8d738f7
feat(client/controllers/recent-imports): Move recent-imports controll…
shivam-tripathi Aug 10, 2018
e563583
feat(client/controllers/import-entity): Add controller for import-ent…
shivam-tripathi Aug 10, 2018
2c25f13
refactor(client/helpers/utils): Remove unused function
shivam-tripathi Aug 10, 2018
b68bbab
feat(client/helpers/import-entity): Add import-entity helper function…
shivam-tripathi Aug 10, 2018
8ba3a82
feat(client/import-entities/footer): Add feature to prevent multiple …
shivam-tripathi Aug 10, 2018
560d24b
feat(client/components/pages/import-entities): Add import-entity pages
shivam-tripathi Aug 10, 2018
2598bdf
feat(client/entity-editor/submission-section): Add logic for redirect…
shivam-tripathi Aug 10, 2018
7c07651
feat(server/helpers/middleware): Add middleware to load imports into …
shivam-tripathi Aug 10, 2018
0374e40
feat(server/helpers/importEntityRouteUtils): Add logic to load edit p…
shivam-tripathi Aug 10, 2018
5fd6e9a
feat(server/helpers/utils): Add function to validate importId, could …
shivam-tripathi Aug 10, 2018
cab5b6a
feat(server/routes/import-entity/transform-form): Add module to trans…
shivam-tripathi Aug 10, 2018
b263307
feat(server/routes/import-entity/transform-import): Add module to tra…
shivam-tripathi Aug 10, 2018
a5b18db
feat(server/routes/import-entity/import-routes): Add import route hel…
shivam-tripathi Aug 10, 2018
0987520
feat(server/routes/import-entity): Add entity functions, update index
shivam-tripathi Aug 10, 2018
c8cb4d0
refactor(server/routes): Update routes to reflect import-entities
shivam-tripathi Aug 10, 2018
e450dec
feat(server/routes/import-entity): Add entity routes
shivam-tripathi Aug 18, 2018
762ccd9
feat(client/recent-imports): Add maximum button limit
shivam-tripathi Aug 18, 2018
26d82a7
feat(components/parts/search): Remove inline styles, add break before…
shivam-tripathi Aug 19, 2018
18fc335
feat(client/components/pages/entities): Export all attributes functions
shivam-tripathi Aug 19, 2018
29e3ada
feat(client/commit/pages/import-entities): Reuse attribute functions …
shivam-tripathi Aug 19, 2018
154f76c
feat(client/components/import-entities): Remove inline css
shivam-tripathi Aug 19, 2018
84989c7
feat(discard-import-entity): Remove token mechanism
shivam-tripathi Aug 19, 2018
18199d5
feat(client/import-entity/footer): Change name
shivam-tripathi Aug 19, 2018
35ea8b0
refactor: General refactor, remove redundant code, update credits
shivam-tripathi Aug 19, 2018
2afd166
feat(stylesheets/style.less): Add some style classes
shivam-tripathi Aug 19, 2018
b15a144
feat(client/helpers/import-entity): Add space after label, refactor
shivam-tripathi Aug 19, 2018
b61c993
feat(client/components/search-results): Use labels to denote imports
shivam-tripathi Aug 19, 2018
f8bb16c
feat(client/controllers/import-entity): Improve error handling when l…
shivam-tripathi Aug 19, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@
"serve-favicon": "^2.4.3",
"superagent": "^3.8.2",
"superagent-bluebird-promise": "^4.1.0",
"uuid": "^3.3.2",
"validator": "^9.1.2"
},
"devDependencies": {
Expand Down
9 changes: 9 additions & 0 deletions scripts/build-client-js.sh
Original file line number Diff line number Diff line change
@@ -1,29 +1,38 @@
#!/usr/bin/env bash

pushd static/js
mkdir -p editor entity import-entity
popd
pushd src/client/controllers
cross-env BABEL_ENV="browser" browserify -t [babelify] \
../entity-editor/controller.js \
editor/edit.js \
editor/achievement.js \
editor/editor.js \
entity/entity.js \
import-entity/import-entity.js \
deletion.js \
index.js \
registrationDetails.js \
revision.js \
search.js \
statistics.js \
import-entity/recent-imports.js \
import-entity/discard-import-entity.js \
-p [ factor-bundle \
-o ../../../static/js/entity-editor.js \
-o ../../../static/js/editor/edit.js \
-o ../../../static/js/editor/achievement.js \
-o ../../../static/js/editor/editor.js \
-o ../../../static/js/entity/entity.js \
-o ../../../static/js/import-entity/import-entity.js \
-o ../../../static/js/deletion.js \
-o ../../../static/js/index.js \
-o ../../../static/js/registrationDetails.js \
-o ../../../static/js/revision.js \
-o ../../../static/js/search.js \
-o ../../../static/js/statistics.js \
-o ../../../static/js/import-entity/recent-imports.js \
-o ../../../static/js/import-entity/discard-import-entity.js \
] > ../../../static/js/bundle.js
popd
9 changes: 9 additions & 0 deletions scripts/watch-client-js.sh
Original file line number Diff line number Diff line change
@@ -1,29 +1,38 @@
#!/usr/bin/env bash

pushd static/js
mkdir -p editor entity import-entity
popd
pushd src/client/controllers
cross-env BABEL_ENV="browser" watchify -t [babelify] \
../entity-editor/controller.js \
editor/edit.js \
editor/achievement.js \
editor/editor.js \
entity/entity.js \
import-entity/import-entity.js \
deletion.js \
index.js \
registrationDetails.js \
revision.js \
search.js \
statistics.js \
import-entity/recent-imports.js \
import-entity/discard-import-entity.js \
-p [ factor-bundle \
-o ../../../static/js/entity-editor.js \
-o ../../../static/js/editor/edit.js \
-o ../../../static/js/editor/achievement.js \
-o ../../../static/js/editor/editor.js \
-o ../../../static/js/entity/entity.js \
-o ../../../static/js/import-entity/import-entity.js \
-o ../../../static/js/deletion.js \
-o ../../../static/js/index.js \
-o ../../../static/js/registrationDetails.js \
-o ../../../static/js/revision.js \
-o ../../../static/js/search.js \
-o ../../../static/js/statistics.js \
-o ../../../static/js/import-entity/recent-imports.js \
-o ../../../static/js/import-entity/discard-import-entity.js \
] -o ../../../static/js/bundle.js -dv
popd
2 changes: 1 addition & 1 deletion src/client/components/pages/entities/creator.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const {extractAttribute, getTypeAttribute, getEntityUrl} = entityHelper;
const {Col, Row} = bootstrap;


function CreatorAttributes({creator}) {
export function CreatorAttributes({creator}) {
const type = getTypeAttribute(creator.creatorType).data;
const gender = extractAttribute(creator.gender, 'name');
const beginArea = extractAttribute(creator.beginArea, 'name');
Expand Down
2 changes: 1 addition & 1 deletion src/client/components/pages/entities/edition.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ const {
} = entityHelper;
const {Col, Row} = bootstrap;

function EditionAttributes({edition}) {
export function EditionAttributes({edition}) {
const status = extractAttribute(edition.editionStatus, 'label');
const format = extractAttribute(edition.editionFormat, 'label');
const pageCount = extractAttribute(edition.pages);
Expand Down
8 changes: 6 additions & 2 deletions src/client/components/pages/entities/image.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,12 @@ function EntityImage({backupIcon, imageUrl}) {
}
EntityImage.displayName = 'EntityImage';
EntityImage.propTypes = {
backupIcon: PropTypes.string.isRequired,
imageUrl: PropTypes.string.isRequired
backupIcon: PropTypes.string,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change being made?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Primarily because right now we have a system which is forced to work without isRequired requirement for the props which raises multiple errors when the component loads. I feel either we should should address the issue or get rid of the hard requirement.
The system at present as such is expected to run without the above mentioned props, so it makes sense to me to remove the hard requirement.

imageUrl: PropTypes.string
};
EntityImage.defaultProps = {
backupIcon: '',
imageUrl: ''
};

export default EntityImage;
2 changes: 1 addition & 1 deletion src/client/components/pages/entities/publication.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import React from 'react';
const {getTypeAttribute, getEntityUrl} = entityHelper;
const {Col, Row} = bootstrap;

function PublicationAttributes({publication}) {
export function PublicationAttributes({publication}) {
const type = getTypeAttribute(publication.publicationType).data;

return (
Expand Down
2 changes: 1 addition & 1 deletion src/client/components/pages/entities/publisher.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import React from 'react';
const {extractAttribute, getTypeAttribute, getEntityUrl} = entityHelper;
const {Col, Row} = bootstrap;

function PublisherAttributes({publisher}) {
export function PublisherAttributes({publisher}) {
const type = getTypeAttribute(publisher.publisherType).data;
const area = extractAttribute(publisher.area, 'name');
const beginDate = extractAttribute(publisher.beginDate);
Expand Down
5 changes: 4 additions & 1 deletion src/client/components/pages/entities/relationships.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,10 @@ function EntityRelationships({relationships}) {
}
EntityRelationships.displayName = 'EntityRelationships';
EntityRelationships.propTypes = {
relationships: PropTypes.array.isRequired
relationships: PropTypes.array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change being made?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

};
EntityRelationships.defaultProps = {
relationships: []
};

export default EntityRelationships;
2 changes: 1 addition & 1 deletion src/client/components/pages/entities/work.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const {getLanguageAttribute, getTypeAttribute, getEntityUrl} = entityHelper;
const {Col, Row} = bootstrap;


function WorkAttributes({work}) {
export function WorkAttributes({work}) {
const type = getTypeAttribute(work.workType).data;
const languages = getLanguageAttribute(work).data;

Expand Down
86 changes: 86 additions & 0 deletions src/client/components/pages/import-entities/creator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
* Copyright (C) 2018 Shivam Tripathi
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

import * as bootstrap from 'react-bootstrap';
import * as importHelper from '../../../helpers/import-entity';

import {CreatorAttributes} from '../entities/creator';
import EntityImage from '../entities/image';
import EntityLinks from '../entities/links';
import ImportFooter from './footer';
import ImportTitle from './title';
import PropTypes from 'prop-types';
import React from 'react';
import _ from 'lodash';


const {getImportUrl} = importHelper;
const {Alert, Col, Row} = bootstrap;


function ImportCreatorDisplayPage({importEntity, identifierTypes}) {
const urlPrefix = getImportUrl(importEntity);
return (
<div>
<Row className="entity-display-background">
<Col className="entity-display-image-box text-center" md={2}>
<EntityImage
backupIcon="user"
imageUrl={importEntity.imageUrl}
/>
</Col>
<Col md={10}>
<ImportTitle importEntity={importEntity}/>
<CreatorAttributes creator={importEntity}/>
</Col>
</Row>
<EntityLinks
entity={importEntity}
identifierTypes={identifierTypes}
urlPrefix={urlPrefix}
/>
<hr className="margin-top-d40"/>
<Row>
<Alert
bsStyle="success"
className="text-center font-weight-bold"
>
{`This ${_.startCase(importEntity.type.toLowerCase())} `}
{'has been automatically added. Kindly approve/discard it '}
{'to help us improve our data.'}
</Alert>
</Row>
<ImportFooter
hasVoted={importEntity.hasVoted}
importUrl={urlPrefix}
importedAt={importEntity.importedAt}
source={importEntity.source}
/>
</div>
);
}
ImportCreatorDisplayPage.displayName = 'ImportCreatorDisplayPage';
ImportCreatorDisplayPage.propTypes = {
identifierTypes: PropTypes.array,
importEntity: PropTypes.object.isRequired
};
ImportCreatorDisplayPage.defaultProps = {
identifierTypes: []
};

export default ImportCreatorDisplayPage;
137 changes: 137 additions & 0 deletions src/client/components/pages/import-entities/discard-import-entity.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
/*
* Copyright (C) 2018 Shivam Tripathi
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

import * as bootstrap from 'react-bootstrap';
import {
getImportDiscardUrl, getImportUrl
} from '../../../helpers/import-entity';
import LoadingSpinner from '../../loading-spinner';
import PropTypes from 'prop-types';
import React from 'react';
import request from 'superagent-bluebird-promise';


const {Alert, Button, ButtonGroup, Col, Panel, Row} = bootstrap;

class DiscardImportEntity extends React.Component {
constructor(props) {
super(props);

this.state = {
error: null,
waiting: false
};

this.handleClick = this.handleClick.bind(this);
}

handleClick() {
event.preventDefault();

this.setState({
error: null,
waiting: true
});

const discardUrl = getImportDiscardUrl(this.props.importEntity);
const importUrl = getImportUrl(this.props.importEntity);

request.post(discardUrl)
.then(() => {
window.location.href = importUrl;
})
.catch((res) => {
const {error} = res.body;

this.setState({
error,
waiting: false
});
});
}

render() {
const {importEntity} = this.props;

let errorComponent = null;
if (this.state.error) {
errorComponent =
<Alert bsStyle="danger">{this.state.error}</Alert>;
}

const loadingComponent = this.state.waiting ? <LoadingSpinner/> : null;

const headerComponent = <h3>Confirm Discard</h3>;

const entityName =
importEntity.defaultAlias ?
importEntity.defaultAlias.name : '(unnamed)';

return (
<div>
<h1> Discard Imported Entity </h1>
<Row className="margin-top-2">
{loadingComponent}
<Col md={6} mdOffset={3}>
{errorComponent}
<Panel
bsStyle="danger"
header={headerComponent}
>
We really appreciate your efforts in helping us
improve our database. The {importEntity.type}
<b className="color-red"> {entityName} </b>
has been automatically added to
our records and will be permanently deleted in
case multiple editors find it to be corrupt.
If you’re sure that the {importEntity.type}
<b className="color-red"> {entityName} </b>
should be discarded, please press the confirm
button below. Other wise click cancel to get back
to the imported entity page.
</Panel>
<ButtonGroup justified className="margin-top-2">
<Button
bsStyle="default"
href={this.importUrl}
title="Edit"
>
Cancel
</Button>
<Button
bsStyle="danger"
href="#"
title="Discard"
onClick={this.handleClick}
>
Discard
</Button>
</ButtonGroup>
</Col>
</Row>
</div>
);
}
}

DiscardImportEntity.displayName = 'DiscardImportEntity';
DiscardImportEntity.propTypes = {
importEntity: PropTypes.object.isRequired
};

export default DiscardImportEntity;
Loading